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EXECUTIVE  SUMMARY 


The  overall  objective  of  this  thesis  is  to  assist  the  Office  of  Naval  Research  (ONR)  and  its 
support  of  a  research  effort  undertaken  by  Orrizonte  Sistemi  Navali  (OSN),  an  Italian 
naval  ship  construction  organization.  OSN’s  work  is  focused  on  the  development  of  naval 
ship  dashboards,  capable  of  facilitating  discussion  between  ship  designers  and  decision 
makers.  Centered  on  the  Offshore  Patrol  Vessel  (OPV),  these  dashboards  utilize  a  series 
of  Operational  Evaluation  Models  (OEMs)  and  Ship  Synthesis  Models  to  help  decision 
makers  understand  the  trade-space  involved  with  naval  ship  architecture.  The  Anti- 
Surface  Warfare  (ASUW)  scenario  being  modeled  in  this  thesis  is  a  part  of  the  OEMs, 
along  with  several  other  mission  areas  such  as  Search  and  Rescue  (SAR),  Maritime 
Interdiction  Operations  (MIO),  and  Cost  Estimation.  These  models  feed  the  OEMs  and 
subsequently,  the  dashboard,  which  then  displays  OPV  perfonnance  in  each  mission  area 
for  a  particular  equipment  setup.  This  performance  indication  can  then  be  adjusted  to 
assist  decision  makers  as  to  which  areas  may  need  more  attention,  or  which  areas  can  be 
lowered  in  importance. 

This  thesis  provides  an  evaluation  of  key  performance  factors  of  a  theoretical 
OPV  in  an  ASUW  Scenario.  To  model  the  scenario,  agent  based  modeling  software  was 
used  to  construct  both  the  OP  Vs  and  the  small  boat  swarm  threat  to  create  an 
environment  capable  of  exploring  a  wide  range  of  factors  and  associated  levels.  The  New 
Zealand  developed  agent  based  modeling  software,  Map  Aware  Non-Uniform  Automata 
(MANA)  was  chosen  to  generate  the  agent  based  models  necessary  for  data  collection 
purposes.  The  simulations  conducted  are  divided  into  two  primary  sets.  The  first  set  is  the 
baseline  simulation  which  focuses  on  establishing  interchangeability  with  a  provided 
OSN  model,  as  well  as  detennining  factor  baseline  importance.  The  second  set  of 
simulations  diverges  from  the  baseline  model  to  explore  a  larger  sample  space  to  identify 
possible  emergent  behavior  and  changes  in  factor  importance  and  interactions. 

Establishment  of  interchangeability  between  the  OSN  model  and  the  baseline 
model  generated  in  MANA  is  important  because  it  helps  to  verify  the  work  performed  by 

OSN.  Verification  eliminates  the  possibility  that  the  results  produced  by  the  OSN  model 
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are  a  result  of  sampling  error  or  other  unknown  factors.  Aside  from  verification, 
interchangeability  provides  a  basis  for  follow-on  factor  comparisons.  If  the  models  are 
interchangeable,  then  if  one  model  possessed  an  additional  capability,  the  other  model 
would  be  expected  to  produce  similar  outcomes  if  it  also  had  the  same  capability.  This 
forms  the  basis  of  the  advanced  models  which  follow  the  baseline  MANA  modeling 
effort. 

The  advanced  modeling  effort  initially  focuses  on  creating  a  more  challenging 
scenario  for  the  modeled  OPVs.  The  number  of  opponents  that  the  OPVs  face  has  been 
increased,  as  well  as  the  introduction  of  behaviors  available  in  the  MANA  modeling 
software.  These  behaviors  are  Avoidance  and  Aggression,  and  their  inclusion  in  the 
advanced  scenarios  is  intended  to  show  how  agent  movement  impacts  OPV  performance. 
To  follow  the  increases  in  the  small  boat  threat,  the  OPVs  are  also  enhanced  to  reflect  a 
more  realistic  naval  platform  seen  at  sea  today.  These  changes  are  then  evaluated  to  see 
how  they  impact  the  more  challenging  small  boat  problem. 

Results  and  insights  gained  from  the  baseline  modeling  effort  indicate  that  the 
models  are  interchangeable  with  the  inclusion  of  an  indicator  variable.  The  naval  gun  is 
clearly  the  most  significant  factor  with  nearly  80%  of  all  variance  accounted  for  by  this 
factor  alone.  Following  this  is  the  type  of  surface-to-surface  missile  system  equipped  on 
the  OPV.  Additionally,  the  inclusion  of  the  helicopter  helps  to  provide  tactical  data  to  the 
OPVs,  and  directs  them  to  the  nearest  small  boat  threat  for  quicker  prosecution.  The 
results  gained  from  the  baseline  effort  seemed  to  stress  an  increase  in  OPV  armament  to 
increase  the  response,  with  very  little  impact  from  interaction  terms  or  other  factors. 

The  advanced  models  indicate  a  divergence  from  the  insights  gained  from  the 

baseline  models.  Given  that  the  small  boat  threat  possessed  the  capability  to  either  evade 

or  attack  OPVs,  then  the  speed  and  mast  height  factors,  and  their  interactions,  became 

more  significant  in  relation  to  the  number  of  small  boat  kills.  In  the  enhanced  OPV 

simulation  runs,  the  naval  gun  significantly  dropped  in  importance  when  compared  to  the 

missile  system  since  its  accuracy  now  reflects  historical  performance  when  engaging 

seaborne  targets.  The  most  significant  factor  was  the  type  of  small  boat  being  engaged  by 

the  OPVs,  followed  by  the  equipped  missile  system  on  the  OPVs.  Ultimately  the 
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advanced  models  indicated  that  an  agile  ship  with  good  radar  capabilities  to  feed 
information  to  its  associated  missile  systems  would  be  the  most  capable  at  combating  the 
small  boat  swann  threat. 
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INTRODUCTION 


The  Offshore  Patrol  Vessel  (OPV)  is  one  of  the  fastest- growing  naval  ship 
classifications  in  the  world  today.  As  of  2009,  there  were  23  nations  worldwide  operating 
OP  Vs  with  76  in  production  or  on  order  at  a  value  of  over  $10  billion  according  to  the 
“Offshore  Patrol  Vessels:  Sector  Report,  2009”  (OPV  Sector  Report,  2009).  The 
following  year,  those  figures  grew  to  30  countries  employing  OPVs  with  another  89  in 
production  and  plans  for  98  at  a  value  of  over  $15  billion,  “Offshore  Patrol  Vessels: 
Sector  Report,  2010”  (OPV  Sector  Report,  2010).  This  begs  the  question,  why  the  OPV 
class  and  not  the  destroyer  or  frigate,  which  are  larger  and  generally  more  capable? 
While  frigates  are  regarded  as  the  smallest  class  of  ship  that  can  operate  independently  on 
the  open  ocean,  the  smaller  OPV  requires  less  manning  and  maintenance,  while  still 
adequately  performing  the  same  mission  capabilities  (Mommsen,  2009).  This  results  in 
reduced  costs  for  naval  ship  production  and  maintenance,  which  is  very  appealing  to 
smaller  nations  given  tightening  budget  constraints. 


Figure  1.  Comandante  Class:  ITS  Comandante  Foscari  (P-493) 


Primarily  focused  on  the  Anti-Surface  Warfare  (ASUW)  and  Anti-Submarine 
Warfare  (ASW)  side  of  national  security,  OPVs  fit  into  a  growing  national  need  of  power 
projection  and  military  capability.  This  military  contribution  is  relevant  due  to  national 

waterway  and  resource  disputes  as  neighboring  nations  vie  for  rich  resource  grounds,  be 
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it  fishing,  trade,  or  offshore  drilling.  The  territorial  dispute  in  the  South  China  Sea  is  a 
good  example:  “The  importance  of  the  South  China  Sea  as  a  strategic  passageway  is 
unquestioned.  It  contains  critical  sea  lanes  through  which  oil  and  many  other  commercial 
resources  flow  from  the  Middle  East  and  Southeast  Asia  to  Japan,  Korea,  and  China” 
(Snyder,  1996).  To  assert  a  nation’s  territorial  claims  they  must  have  a  presence  in  the 
region,  and  warships  have  traditionally  been  the  facilitators  of  that  presence.  Aside  from 
power  projection,  OPVs  have  increased  in  popularity  over  the  recent  decades  due  to  their 
capabilities  in  Maritime  Interdiction  Operations  (MIO)  and  Search  and  Rescue  (SAR), 
which  predominately  fall  into  the  roles  of  the  coast  guard.  Unlike  the  U.S.,  which 
separates  its  coastal  and  naval  forces,  many  nations  utilize  their  navies  to  fulfill  both 
roles.  OPVs  are  a  great  asset  for  this  capability,  and  for  the  smaller  nation,  modem  OPVs 
provide  an  adequate  projection  of  naval  power  at  a  relatively  cheap  price. 

The  OPV  is  also  appealing  due  to  its  reduced  manufacturing  costs  when  compared 
to  other  combatants.  OPVs  are  generally  built  in  two  separate  fashions,  the  first  being  a 
higher  cost  military  variant  stressing  weapon  systems,  while  the  other  is  for  generally 
more  peacetime  operations.  The  second  variant  is  aimed  toward  commercial  and  civilian 
specifications  rather  than  military  ones  to  reduce  costs.  The  reason  behind  this  shift  is  to 
help  ease  the  fabrication  and  system  operating  requirements  that  military  specifications 
(MILSPEC)  impose  on  naval  vessels  and  designers.  Considerations  as  to  ship  damage 
survivability  and  system  redundancy  are  stressed  more  so  in  MILSPEC  vessels  than 
commercial  vessels.  Commercially  oriented  ships  tend  to  focus  more  on  crew  habitability 
and  survivability,  thus  creating  a  cheaper  platform  to  produce.  Thus  for  a  nation  with 
tightened  budgets  for  military  spending,  the  OPV  provides  flexibility  given  its  prescribed 
mission. 
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A.  BACKGROUND 

During  the  summer  of  2008,  Ms.  Kelly  Cooper,  Program  Officer  at  the  Office  of 
Naval  Research  (ONR),  attended  the  EURO  SIW,  International  Simulation  Multi- 
Conference  concerning  innovative  studies  applied  for  the  estimation  of  naval  ship 
effectiveness.  It  was  at  this  conference  that  Ms.  Cooper  met  Dr.  Natalino  Dazzi,  a  project 
manager  of  Orizzonte  Sistemi  Navali  (OSN),  which  is  a  leader  in  Italian  warship 
operational  design  and  production  with  ties  to  other  international  markets.  Dr.  Dazzi  and 
his  OSN  staff  were  presenting  their  work  on  research  activities  and  innovative  studies 
applied  for  estimation  of  the  naval  vessel  effectiveness  in  regards  to  mission  task,  cost, 
and  perfonnances.  Interested  in  this  topic,  Ms.  Cooper  speculated  whether  the  same 
efforts  could  be  applied  to  current  U.S.  ship  modeling  efforts.  A  dialog  between  the  two 
was  created  and  in  early  December  2010,  a  workshop  was  held  in  Rome  with  ONR  and 
OSN,  to  begin  a  project  focused  on  techniques  and  tools  to  facilitate  discussion  between 
decision  makers  on  naval  ship  design,  particularly  on  an  Italian  OPV.  The  project  was 
titled  the  Application  System  for  Naval  Evaluation  and  Testing  (ASNET),  Partnership  for 
Research  on  Naval  Technology  and  Operations  (PRONTO),  Naval  International 
Cooperative  Opportunities  in  Science  &  Technology  Program  (NICOP).  Its  mission  is  the 
linking  of  the  Operational  Evaluation  Model  (OEMs)  and  Ship  Synthesis  Model  to  better 
provide  infonnation  for  customers  and  streamline  this  ship  building  process.  ONR’s 
interest  is  the  possibility  of  broadening  this  research  for  use  in  the  U.S.  Navy  and  its  own 
ship  construction  endeavors.  The  Naval  Postgraduate  School  (NPS),  Georgia  Institute  of 
Technology  (GT),  Virginia  Polytechnic  Institute  of  Technology  (VT),  and  several  other 
organizations  have  been  brought  in  to  work  on  this  project. 

Tapped  as  the  project  lead  for  the  NPS  effort  in  the  ASNET/  PRONTO  NICOP, 
Dr.  Eugene  Paulo,  of  the  Systems  Engineering  Department,  identified  the  need  for 
operations  analysts  to  be  included  in  the  endeavor.  This  analytical  role  was  filled  by 
several  Operations  Research  (OR)  master’s  students  and  Research  Assistants  (RA),  as 
well  as  faculty  from  the  Simulation  Experiments  &  Efficient  Designs  (SEED)  Center  for 
their  simulation  and  computational  capabilities.  Each  OR  master’s  student  would 
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approach  a  particular  mission  area  of  the  simulated  OPV,  conduct  analysis  on  the  OSN 
model  outputs,  and  expand  their  models  to  encompass  further  research  questions.  The 
mission  areas  addressed  were  Anti-Surface  Warfare,  Maritime  Interdiction  Operations, 
Search  and  Rescue,  and  Cost  Estimation.  With  the  exception  of  the  cost  estimation,  each 
OR  student  was  provided  with  an  OSN  generated  agent  based  model  and  its  associated 
data  to  form  a  basis  for  their  research  efforts. 

B.  RESEARCH  QUESTIONS 

This  thesis  is  based  on  the  simulation  modeling  of  a  hypothetical  OPV,  to  provide 
insightful  information  to  decision  makers  on  naval  ship  design  and  procurement.  This 
insight  is  gained  by  the  generation  of  dashboards  that  receive  input  from  their  associated 
models.  The  dashboard  currently  in  development  at  the  Naval  Postgraduate  School  is 
focused  on  the  linking  of  naval  operational  models  to  ship  synthesis  models.  Prior  to  their 
inclusion  into  the  dashboard  however,  the  models  would  be  required  to  perfonn  in  such  a 
manner,  as  to  produce  results  that  are  both  believable  and  realistic.  The  models  developed 
in  the  course  of  this  thesis  are  focused  on  answering  the  following  questions: 

1 .  Based  on  the  limited  number  of  design  parameters  of  the  OPV  in  the  OSN 
model,  can  a  model  be  produced  in  the  Map  Aware  Non-Uniform 
Automata  (MANA)  (McIntosh  et  ah,  2012)  environment  to  replicate 
similar  results? 

2.  Given  the  creation  of  this  initial  MANA  model  of  the  OPV’s  ASUW 
scenario,  can  additional  variables  be  introduced  to  determine  a  better 
Operational  Evaluation  Model  for  OPV  perfonnance  and  subsequent  input 
into  the  Ship  Synthesis  Model  and  cost  models? 

C.  SCOPE  OF  THE  THESIS 

This  thesis  focuses  on  the  Anti-Surface  Warfare  scenario  presented  by  the  OSN 

modeled  OPV.  Guided  by  the  previously  presented  research  questions,  the  modeling 

effort  first  focused  on  the  creation  of  the  OSN  ASUW  scenario  in  the  MANA 

environment.  Second,  once  model  behavior  was  established  to  be  behaving  adequately 

compare  to  the  OSN  agent  based  model,  additional  factors  were  introduced  to  gain 
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further  insight  on  the  importance  of  key  factors  in  the  OPV  design.  MANA  version  5  was 
utilized  to  model  both  the  baseline  scenario  and  more  advanced  scenarios.  Analysis  of  the 
baseline  scenario  would  need  to  show  its  interchangeability  with  the  OSN  agent  based 
model,  as  well  as  establish  a  benchmark  for  key  performance  factors  of  the  modeled 
OPVs.  The  advanced  models  analysis  ascertains  if  these  key  performance  factors  change 
in  the  new  scenario  environments,  and  identify  areas  of  investment  to  further  enhance 
OPV  technologies.  Once  the  analysis  is  completed,  the  conclusion  and  recommendation 
portion  of  this  thesis  explains  why  these  changes  occurred,  and  present  insights  as  to 
optimal  OPV  configurations  for  the  ASUW  scenario. 

D.  THE  ASYMMETRIC  THREAT 

While  the  specifics  of  the  scenario  are  detailed  later,  the  overarching  problem  that 
is  addressed  in  these  models  is  the  small  boat  swarm  attack.  Research  work  concerning 
this  topic  has  been  of  increasing  frequency  given  the  asymmetric  tactics  employed  by 
terrorist  organizations  and  militaries  still  operating  in  the  world  today.  The  attack  on  the 
USS  COLE  on  October  12,  2000,  while  she  conducted  refueling  operations  in  Yemen  is  a 
stark  reminder  of  how  effective  a  single  small  boat  laden  with  explosives  can  be  against  a 
ship. 


Figure  2.  USS  Cole  (DDG-67)  Being  Towed  to  its  Rendezvous  With  M/V  Blue 

Marlin 

Another  example  of  effective  asymmetric  tactics  is  the  Sri  Lankan  Civil  War.  The  Tamil 
Sea  Tigers,  utilizing  asymmetric  tactics  such  as  suicide  boats  and  swarming  maneuvers, 
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were  effective  in  engaging  in  a  civil  war  for  almost  three  decades,  where  they  inflicted 
heavy  loses  to  the  government  sponsored  Sri  Lankan  Navy.  Only  by  enhancing  strategies 
and  military  capabilities  of  its  navy  was  Sri  Lanka  able  to  claim  victory  over  the 
Liberation  Tigers  of  Tamil  Eelam  (LTTE)  on  May  17,  2009.  This  victory  was  largely  due 
to  the  effective  isolation  of  LTTE  strongholds  by  cutting  all  sea  lanes  of  communication 
(SLOC),  thus  preventing  outside  resupply  of  resources  and  weapons.  To  do  this,  Sri 
Lankan  naval  forces  employed  fast  attack  craft  and  OPVs  to  establish  multiple  barriers  of 
defense  to  both  defend  from  Sea  Tiger  attack  swarms  and  prevent  retreating  LTTE  forces 
from  escaping  (Fish,  2009). 

Apart  from  actual  conflict,  study  of  Iran’s  naval  doctrine  against  future  adversaries 
has  shown  that  asymmetric  tactics  will  be  the  focus  to  defeat  larger,  more  capable  outside 
aggressors  in  naval  engagements,  such  as  the  U.S.  Navy.  In  wartime,  Iranian  naval  forces 
would  seek  to  close  the  Strait  of  Honnuz  and  destroy  forces  bottled  up  in  the  Persian 
Gulf;  therefore  speed  and  surprise  would  be  key  (Haghshenass,  2006).  To  achieve  this, 
Iran  has  been  focusing  on  the  acquisition  and  development  of  small,  fast  weapons 
platfonns,  particularly  lightly  anned  small  boats  and  missile  anned  fast  attack  craft 
(Haghshenass,  2006).  Additionally  Iran  has  been  encouraging  decentralized  decision 
making  practices  in  its  naval  forces,  to  further  enhance  and  support  its  naval  swarm 
tactics. 

All  of  the  previous  examples  present  a  very  real  and  current  threat  that  has  to  be 
addressed  by  the  navies  of  the  world  today.  Presuming  a  navy  adopts  strategies  to  combat 
swanning  threats  and  other  asymmetric  tactics,  the  next  question  would  be  as  to  what 
technology  would  best  combat  the  problem.  Contenders  would  surely  be  small,  fast, 
patrol  craft  that  can  cover  range  at  speed  and  in  volume.  Additionally  they  would  have  to 
be  supported  by  persistent,  wide-area  surveillance  systems,  most  likely  airborne  in  nature 
(Mallon,  2011).  The  OPV  is  one  of  the  ship  classes  that  can  meet  these  requirements. 
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E.  SCENARIO  OVERVIEWS 

1.  Baseline  Scenario 

The  OSN  provided  scenario  consists  of  an  established  combat  area  where  three 
OPVs  perform  a  defensive  role  against  approaching  Small  Fast  Attack  Craft  (SFAC).  It  is 
the  mission  of  these  SFACs  to  reach  a  goal  line  which  is  protected  by  the  three  OPVs. 
The  OPVs  are  equipped  with  a  varying  array  of  weapons  and  capabilities  to  employ  in 
this  goal  line  defense.  The  type  of  SFACs  and  their  associated  starting  area  are  also 
varied  to  create  a  random  battleground  for  the  OPVs  to  perform  in. 

The  specific  capabilities  of  the  OPVs  that  are  varied  include  missile  type 
equipped,  the  presence  of  a  naval  gun,  maximum  speed,  and  mast  height  which  impacts 
the  detection  and  classification  range  of  the  OPV.  Weapon  hit  probabilities  and  ranges 
are  fixed,  and  are  specific  to  the  associated  weapon.  Additional  to  the  ship  capabilities, 
the  presence  of  a  helicopter,  which  assists  the  OPVs  by  detecting  and  classifying  SFACs 
and  relaying  target  data,  is  introduced  to  half  of  the  modeled  scenarios.  The  OPV  patrol 
routes,  starting  locations,  and  patrol  speeds  are  fixed  for  the  initial  modeling  effort. 
OPVs,  and  the  helicopter  if  present,  relay  targeting  data  perfectly  with  an  unlimited 
communication  range.  If  SFACs  are  detected,  the  OPVs  accelerate  to  maximum  speed 
and  attempt  to  intercept  and  destroy  them.  In  the  absence  of  SFAC  detection,  OPVs 
return  to,  or  continue,  their  established  patrol  route  at  the  prescribed  patrol  speed. 

The  SFACs  varied  parameters  focus  on  their  max  speed  and  size,  which  impacts 
the  range  at  which  OPVs  and  the  helicopter  can  detect  and  classify  them.  SFACs  are 
randomly  generated  in  a  home  box  in  the  modeled  environment,  and  the  probability  of 
their  random  location  distribution  in  the  box  is  uniform.  There  are  nine  total  SFACs  in 
the  scenario,  and  for  the  initial  modeling  efforts,  this  is  a  fixed  parameter.  SFAC  behavior 
is  also  fixed,  in  that  they  proceed  to  the  goal  line  at  max  speed  on  the  shortest  route 
possible.  They  are  not  influenced  by  outside  factors  such  as  the  OPVs  or  nearby  SFACs. 
SFACs  hit  by  OPV  weapons  are  eliminated  from  the  battle,  and  this  infonnation  is 
gathered  by  the  OPVs  instantaneously. 
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2. 


Advanced  Scenarios 


In  the  effort  to  better  understand  the  importance  of  key  factors  detennined  in  the 
baseline  scenarios,  the  modeling  space  has  been  increased  to  accommodate  additional 
behaviors  and  factors.  The  reason  is  to  see  if  these  previously  detennined  factors  change 
in  importance,  or  if  other  factors  become  more  significant.  Increasing  the  realism  of  the 
baseline  scenario  has  to  be  done  with  caution  however,  as  it  may  produce  results  that  may 
be  misleading  or  shed  no  useful  infonnation  to  the  problem  of  the  OPV  effectiveness.  It 
can  be  problematic  if  particular  factors  are  varied  individually  which  result  in  no 
significant  change  to  the  response  of  the  advanced  model.  This  can  lead  to  the  conclusion 
that  these  factors  are  not  important,  yet  it  may  be  true  that  the  interactions  of  these  varied 
factors  cause  a  significant  change  in  the  response.  Sanchez  (2006)  uses  capture-the-flag 
as  an  example,  where  speed  and  stealth  are  varied  to  their  high  and  low  set  points,  but  are 
not  varied  jointly.  The  end  result  is  that  neither  factor  changed  the  response  by  itself.  If 
the  interaction  of  speed  and  stealth  was  not  explored,  then  a  reader  may  conclude  that 
neither  speed  nor  stealth  will  help  in  capture-the-flag,  while  direct  observation  says 
otherwise  when  both  are  varied  together. 

For  the  advanced  scenarios,  addressing  the  behavior  of  the  SFACs  was  of  interest 
to  the  OSN  staff  when  this  thesis  was  in  its  initial  phases  of  development.  They  were  very 
interested  in  the  possibility  of  varying  tactics  that  SFACs  could  take  as  they  approached 
the  OPVs,  and  to  see  the  results  of  the  OPV  response.  The  strategy  to  investigate  this 
increase  in  SFAC  behavior  is  a  stepped  process.  First,  the  avoidance  behavior  is 
introduced  to  the  SFACs,  which  means  they  now  can  detect  and  attempt  to  avoid  OPVs 
as  they  proceed  toward  the  goal  line.  Second  is  the  aggressive  behavior,  where  SFACs 
detect  and  attempt  to  attack  OPVs  utilizing  a  suicide  weapon.  Finally,  a  combination  of 
both  avoidance  and  aggressive  SFACs  are  introduced  for  OPVs  to  prosecute.  The  goal  for 
this  scenario  is  to  see  if  OPVs  are  drawn  away  or  otherwise  preoccupied  to  prevent 
avoidance  SFACs  from  reaching  the  goal  line.  Once  these  scenarios  are  run,  another  set 
of  the  same  scenarios  are  conducted,  but  with  OPVs  with  enhanced  capabilities  more  in 
line  with  real  world  situations.  Many  of  these  capabilities  are  drawn  from  previous  theses 
such  as  Abel’s  (2009)  study  on  frigate  defense  effectiveness  in  asymmetric  environments. 
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In  each  of  these  scenarios,  the  number  of  SFACs  has  been  doubled  from  nine  to  eighteen. 
The  reason  for  this  change  is  to  stress  the  OPV’s  ability  to  defend  the  goal  line,  and  to 
refocus  the  study  from  the  measure  of  effectiveness  of  the  OPVs  of  successful  goal  line 
defense,  to  the  measure  of  performance  of  weapon  and  system  effectiveness. 

F.  CHAPTER  OUTLINE 

Chapter  II  is  the  literature  review  portion  of  the  thesis,  and  it  provides  insight  on 
current  published  references  that  have  greatly  impacted  this  research  effort. 

Chapter  III  contains  the  methodology  and  data  portion  of  this  study.  It  details 
specific  parameters  in  the  models  created,  as  well  as  differences  present  between  the 
provided  OSN  agent  based  model  and  the  NPS  generated  MANA  model.  Also, 
limitations  and  assumptions  were  addressed,  as  well  as  the  measures  of  effectiveness  for 
the  scenario. 

In  Chapter  IV,  produced  data  was  analyzed  to  gather  and  understand  important 
infonnation  and  outcomes.  Initially  this  chapter  establishes  the  interchangeability  of  the 
OSN  model  to  the  NPS  baseline  model,  and  identifies  which  parameters  are  of  the  most 
significance  for  OPV  perfonnance.  Following  this,  the  advanced  model  analysis 
ascertains  whether  or  not  previously  identified  significant  factors  remain  so,  or  if  other 
factors  become  more  noteworthy. 

Chapter  V  is  the  conclusions  and  recommendations  portion  of  this  study.  It 
provides  the  overview  of  the  analyzed  data,  and  produces  recommended  OPV 
configurations  to  meet  the  ASUW  scenario.  Additionally,  it  provides  possibilities  of 
future  work  to  help  expand  and  grow  this  analysis  to  encompass  a  larger  sample  space, 
and  more  realistic  factor  set  points. 

G.  BENEFITS  OF  THE  STUDY 

This  thesis  is  in  support  of  the  ASNET/PRONTO  NICOP  Project  as  part  of 
ONR’s  support  of  the  OSN  OPV  design  effort.  The  modeling  and  analysis  conducted 
helps  to  provide  a  better  picture  as  to  key  design  parameters  in  the  ASUW  role  of  the 
developing  OPV.  Additionally,  the  work  conducted  may  help  to  identify  areas  of  concern 
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or  technology  gaps  to  invest  in  prior  to  actual  ship  construction.  Upon  completion  of  the 
project,  analytical  processes,  and  ship  design  considerations  will  be  utilized  by  ONR  to 
help  in  further  U.S.  naval  ship  modeling  and  construction  efforts. 
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II.  LITERATURE  REVIEW 


A.  INTRODUCTION 

This  chapter  covers  the  principles  and  practices  that  are  most  relevant  to  the  OPV 
modeling  situation  and  how  published  works  contributed  to  this  study.  Since  this  thesis 
revolves  around  a  naval  surface  warfare  scenario,  it  is  important  to  analyze  established 
naval  surface  warfare  doctrine  and  how  it  applies.  Additionally,  the  dilemma  being 
addressed  in  the  modeling  environment,  the  small  boat  swann  attack,  is  a  current  and 
realistic  threat  to  both  naval  ships  and  strategic  national  assets,  such  as  ports,  oil  derricks, 
and  vital  shipping  lanes.  There  is  a  wealth  of  published  works  regarding  this  threat,  and 
this  thesis  draws  on  several  theses  regarding  the  strategy.  Understanding  the  principles 
used  in  agent  based  modeling  is  important  as  well,  and  this  thesis  references  and  expands 
upon  previous  works  utilizing  many  of  the  same  techniques.  Finally,  the  overall  effort  of 
the  ASNET/PRONTO  NICOP  project  is  intended  to  facilitate  discussion  and  give 
direction  in  naval  ship  production,  therefore  understanding  historical  practices  in  ship 
building  is  important. 

B.  NAVAL  WARFARE  DOCTRINE 

Before  any  topic  concerning  naval  doctrine  can  be  written,  a  firm  grasp  as  to  the 
specific  context  of  the  word  “doctrine”  is  important.  The  word  “doctrine”  can  be  defined 
as  a  set  of  beliefs  which  are  held  and  taught  by  an  associated  group,  such  as  a  church, 
government,  or  other  such  organization.  The  term  naval  doctrine  then  implies  that  it  is  a 
set  of  beliefs  held  and  taught  concerning  naval  tactics  and  practices  by  a  navy.  Vice 
Admiral  Luigi  Donolo,  in  his  publication,  The  History  of  Italian  Naval  Doctrine  (1995) 
defines  it  as: 

...the  set  of  principles,  rules  (and  also  beliefs  and  values)  which  indicate 
what  that  navy  must  be,  who  or  what  it  must  represent,  how  it  must  behave 
and  for  what  future  perspectives  it  must  prepare  itself. 

The  U.S.  Navy’s  definition  in  Naval  Doctrine  Publication  1:  Naval  Warfare  (1994)  is 
similar: 
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Naval  doctrine  is  the  foundation  upon  which  our  tactics,  techniques,  and 
procedures  are  built.  It  articulates  operational  concepts  that  govern  the 
employment  of  naval  forces  at  all  levels. 

Other  nations  also  have  comparable  definitions  for  their  naval  forces,  and  the  underlining 
principle  gathered  from  these  doctrines,  is  that  they  are  forged  from  a  nation’s  historical 
actions  at  sea,  and  the  impacts  that  they  have  had  on  military  and  political  interests. 

C.  SWARM  TACTICS 

The  term  “swarming”  is  defined  as  the  act  of  besetting  or  surrounding  in  a  swarm. 
The  noun  “swarm”  is  defined  as  a  large  group  of  things,  usually  in  motion.  Thus  it  is  the 
act  of  using  motion,  or  speed,  to  surround  something  with  a  large  number  of  other  things. 
When  related  to  combat  studies  and  tactics,  swarming  occurs  when  several  units  conduct 
a  convergent  attack  on  a  target  from  multiple  axes  (Edwards,  2004).  Historical  records 
show  that  some  of  the  first  military  efforts  employing  swanning  tactics  date  back  to  the 
time  of  Alexander  the  Great  and  Genghis  Khan,  where  bowmen  on  horseback  would 
swann  and  encircle  their  opponents.  Engagements  in  that  time  were  made  more  difficult 
due  to  the  need  for  quick  and  reliable  communication  between  fighting  elements.  This  is 
especially  true  with  regards  to  coordinating  swann  attacks  due  to  the  large  number  of 
elements  and  decentralized  nature  of  command  and  control  needed  for  the  attack. 
Swarming  has  become  even  more  prevalent  in  modem  times  due  to  its  asymmetric 
nature,  and  partially  because  of  the  proliferation  of  mobile  communication  devices  which 
helps  to  ease  the  coordination  problem.  The  Battle  of  Mogadishu  in  October  1993  is  a 
good  example  of  swarming  tactics  being  employed  by  an  under-trained,  under-equipped 
force  utilizing  simple  coordination  tactics  to  overcoming  a  superior  trained  and  equipped 
enemy.  For  the  focus  of  this  thesis  however,  the  small  boat  swarm  attack  is  the 
overarching  problem  to  be  addressed. 

Cobian  (2002)  studied  the  possible  use  of  the  Javelin  weapon  system  for  naval 
ship  defense  against  small  boat  attack.  He  found  that  the  weapon  system  possessed  very 
high  marks  for  probability  of  hit  and  kill  at  relatively  low  costs  to  eliminate  a  small  boat 
threat. 
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ISSUES  /  CONCERNS 
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LI  MITE 
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N 

N 

Y 
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N 

NEGLIGIBI 

Machine  Guns 

0.2 

0.2 

0.0 

N 

N 

Y/N 

NONE 

Y 

NEGLIGIBI 

Small  Ams 

0.0 

0.0 

0.0 

N 

N 

N 

NONE 

Y 

NEGLIGIBI 

Javelin 

0.9 

0.9 

0.8 

N 

N 

N 

NONE 

Y 

$65,000 

Table  1.  LT.  Cobian’s  Relative  Comparison  of  Weapon  Systems  (Cobian,  2002). 


Additionally,  he  showed  the  ineffectiveness  of  shipboard  weapons  employed  today  at 
preventing  a  small  boat  swarm  attack.  The  results  are  disheartening,  especially  since  the 
next  best  weapon  system  at  prosecuting  an  attack  was  the  Close-In  Weapon  System 
(CIWS),  which  requires  shipboard  power  and  interlocks  controlled  by  the  Commanding 
Officer  (CO)  to  be  disabled  prior  to  use.  The  original  purpose  of  the  CIWS  is  for  ship 
defense  from  sea-skimming  anti-ship  missiles.  Recent  modifications  have  been  made  to 
help  broaden  its  use  against  low  elevation  fliers  such  as  helicopters  and  surface  threats, 
but  these  features  require  operator  control,  and  do  not  utilize  the  automated  features  of 
CIWS.  Additionally,  there  may  be  restrictions  on  both  radar  emissions  or  gun  traversing 
while  entering  or  staying  in  port,  both  of  which  limit  the  CIWS’  effectiveness.  LT. 
Cobian’s  work  showed  the  dangerously  vulnerable  state  warships  are  placed  in  when 
pulling  into  port,  not  just  from  its  weapon  limitations,  but  due  to  its  proximity  to  land. 
Warships  when  in  port  are  limited  in  their  ability  to  maneuver  due  to  traffic  or  shoal 
water.  Given  the  ease  at  which  a  swarm  can  hide  in  a  port,  a  ship  is  definitely  in  a 
disadvantageous  situation. 

Apart  from  the  pier-side  situation  that  the  USS  Cole  faced  in  Yemen,  swarm 
attacks  at  sea  are  also  of  great  concern.  Abel’s  (2009)  work  on  analyzing  frigate 
defensive  effectiveness  showed  that  in  the  green  water  environment,  speed  and 
maneuverability  can  be  just  as  effective  as  having  enhanced  weapon  systems  at 
preventing  a  successful  swarm  attack.  Increasing  the  speed  of  his  modeled  Fast  Frigate 
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with  Helicopter  (FFH)  allowed  it  to  outmaneuver  and  outrun  swarming  SFACs.  This 
conclusion  correlates  with  some  established  naval  doctrine  concerning  small  boat  attacks. 
The  stiff-arm  tactic,  for  example,  consists  of  accelerating  to  full  speed  toward  open-ocean 
while  engaging  crew-served  weapons  on  pursuing  SFACs.  The  reason  behind  this  is  that 
small  boats  are  more  susceptible  to  ocean  sea  states,  and  are  hindered  in  aiming 
ordinance  at  an  avoiding  vessel.  If  everything  goes  well,  the  SFACs  may  break  off  the 
attack  and  return  to  their  base  of  operations  or  be  rendered  inoperable. 


Figure  3.  Photo  of  Iran’s  Great  Prophet  5  Small  Boat  Exercise 


Utilizing  evasive  maneuvers  and  tactics  can  have  significant  results  when 
avoiding  the  small  boat  attack,  be  it  armed  assault  or  piracy  as  outlined  in  Lieutenant 
Commander  (LCDR)  William  Major’s  article  on  assessing  counter-piracy  tactics.  Here 
LCDR  Major  showed  an  increase  in  effectiveness  of  36%  of  avoiding  a  successful  piracy 
boarding  by  commercial  ships  employing  evasion  maneuvers  alone  (Major,  2012). 

While  enhancing  the  ship’s  capabilities  and  weapons  are  obvious  routes  to 
improve  naval  ship  survivability  in  the  small  boat  swarm  environment,  including  a 
shipboard  helicopter  further  increases  that  effectiveness.  While  Kapitanleutnant  Abel’s 
work  did  investigate  the  benefits  of  shipboard  helicopters,  Turkish  Navy  Lieutenant 
Junior  Grade  (LTJG)  Omur  Ozdemir  explored  the  capabilities  of  several  variants  of 
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surface  combatants.  In  his  study,  he  found  that  the  U.S.  Navy’s  new  LCS  class  of  ships 
perfonned  very  well  due  to  its  capability  of  carrying  two  anned  helicopters.  This  feature, 
in  a  non-Surface  to  Air  Missile  (SAM)  environment,  allowed  his  modeled  LCS  ships  to 
detect  and  engage  the  SFAC  threat  far  earlier  (Ozdemir,  2009).  This  allowance  of  time  is 
vital  since  it  can  allow  the  ship  to  plan  and  prepare  for  an  attack,  which  is  the  antithesis  to 
the  swanning  strategy.  Surprise  and  mobility  are  the  cornerstones  of  the  swann  attack,  as 
outlined  in  Iran’s  Doctrine  of  Asymmetric  Warfare  (Haghshenass,  2006).  Therefore,  a 
possible  way  to  combat  the  swanning  threat  is  early  detection,  which  is  greatly  enhanced 
when  patrolling  aircraft  are  employed. 

D.  AGENT  BASED  MODELING 

With  the  propagation  of  widespread  computers  and  computational  resources,  the 
concept  of  agent  based  modeling  has  seen  a  boom  in  popularity  over  the  past  two 
decades.  Their  applications  are  extensive,  with  examples  ranging  from  observing 
financial  market  behaviors  to  agents  conducting  counter  terrorism  operations  in  modeled 
environments.  Their  origins  can  be  traced  back  to  the  1940s  with  the  Von  Neumann 
Machine  which  consisted  of  a  theoretical  machine  that  would  replicate  itself  based  on  a 
prescribed  series  of  processes.  While  it  is  a  far  cry  from  the  modeling  software  used 
today,  it  created  the  basis  for  simulation  modeling.  It  is  still  hard  to  identify  the  first  time 
that  the  phrase  “agent  based”  modeling  was  created,  but  some  sources  indicate  that  the 
tenn  was  coined  by  John  Miller’s  and  John  Holland’s  work  “Artificial  Adaptive  Agents 
in  Economic  Theory”  (Miller  and  Holland,  1991) 

One  of  the  most  notable  studies  conducted  in  early  agent  based  modeling  was 
Craig  Reynold’s,  “Flocks,  Herds,  and  Schools:  A  Distributed  Behavioral  Model.”  This 
work  focused  on  the  establishment  of  modeled  entities,  called  boids,  which  were 
programmed  with  set  behavioral  urges  with  regards  to  the  other  boids  around  them.  First 
was  the  desire  to  stay  near  the  center  of  the  flock,  which  in  general,  means  boids  want  to 
fly  near  other  boids.  The  second  was  collision  avoidance,  in  that  boids  did  not  want  to 
come  into  contact  with  nearby  boids  and  other  obstacles.  Finally,  the  urge  to  match  speed 
with  other  boids  was  instilled  in  the  entities.  This  would  create  three  separate  signals  in  a 
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three  dimensional  vector  space  trying  to  dictate  boid  flight  paths.  The  boid  navigational 
module  would  receive  these  inputs,  and  try  to  generate  an  optimized  path  to  follow  based 
on  these  signals  and  their  associated  weighting  factors. 


Figure  4.  Graphical  Simulation  of  Boids  in  Flight 


This  work  was  one  of  the  first  attempts  to  model  behavioral  responses  by  simulated 
agents  since  not  only  did  they  react  to  themselves,  they  also  reacted  to  their  environment 
when  obstacles  were  placed  in  their  path.  This  modeling  effort  led  to  much  of  the  work 
produced  today  since  the  real  essence  of  agent  based  modeling  is  trying  to  identify 
emergent  behavior  which  can  lead  to  new  and  unexpected  answers. 

The  notion  of  behavioral  response  in  agent  based  modeling  was  addressed  in 
Macy  and  Wilier  (2002).  They  were  trying  to  understand  the  sociological  influences  in  an 
agent  based  modeling  environment  and  the  discovery  of  emergent  behavior  of  agents 
placed  in  differing  sociologic  situations.  While  agent  based  models  don’t  necessary  aim 
to  give  specific  results  to  a  study,  they  do  try  to  gain  further  understanding  of  the 
processes  that  may  appear.  Unlike  Craig  Reynold’s  work  which  focused  on  the  response 
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of  a  single  boid  as  it  flew  in  its  flock,  the  work  performed  by  Macy  and  Wilier  was  to 
understand  how  the  different  social  networks  involved  impacted  the  social  behavior  of 
the  agents.  When  discussing  emergent  order  and  relational  stability  they  address  how,  in 
the  Prisoner’s  Dilemma,  the  optimal  choice  for  an  individual  play  is  defection.  This 
choice  does  not  hold  up  however  when  repeated  play  and  relationships  are  introduced.  At 
this  point,  it  was  impossible  to  predict  the  outcome  of  the  game  because  the  social 
behaviors  of  the  agents  influence  the  choice  of  when  to  defect  or  cooperate.  Ultimately 
the  Tit-for-Tat  strategy  proved  to  be  the  most  advantageous  since  it  always  cooperated 
until  it  was  betrayed,  but  responded  to  betrayal  with  one  time  betrayal  followed  by 
forgiveness.  At  the  end  of  their  study,  they  concluded  that  agent  based  models  presented 
an  extensive  field  of  study  for  sociologists  given  its  behavioral  study  applications. 

E.  DESIGN  OF  EXPERIMENTS 

As  previously  mentioned,  the  OSN  data  collection  effort  for  their  study  involved 
utilizing  factorial  designs.  While  factorial  designs  are  definitely  useful  for  simulation 
experiments,  when  the  number  of  factors  increases,  or  the  number  of  levels  for  a 
particular  factor  increase,  then  more  efficient  designs  may  be  required.  In  Sanchez  and 
Wan  (2009)  they  show  how  quickly  both  course  and  fine  factorial  designs  become 
cumbersome  and  computationally  expensive  as  more  factors  or  levels  are  introduced. 
Since  factorial  designs  produce  situations  of  exponential  growth  in  data  requirements  as 
the  number  of  factors  increase,  their  use  has  to  be  limited  to  experiments  with  low 
numbers  of  factors.  Their  solution  to  this  computational  growth  problem  is  the  use  of 
design  strategies  such  as  Central  Composites  and  Nearly  Orthogonal  Latin  Hypercubes 
(NOLH).  Both  of  these  strategies  attempt  to  establish  a  space-filling  design  that  provides 
insight  onto  the  multidimensional  space  of  an  experiment  which  may  have  been  too  large 
for  a  factorial  design.  The  NOLH  utilized  in  the  data  collection  process  of  this  thesis  is 
the  current  evolution  of  a  number  of  previous  works  concerning  DOE.  Cioppa  (2002), 
Hernandez  (2008),  and  Vieira  et  al.  (2011)  formed  the  current  practices  in  DOE, 
particularly  Nearly  Orthogonal  Latin  Hypercubes  and  space  filling  designs. 
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III.  METHODOLOGY 


A.  INTRODUCTION 

It  is  important  to  first  clarify  that  the  processes  for  attaining  the  baseline  model 
and  advanced  models  are  different,  and  the  questions  that  they  are  intended  to  answer  are 
also  dissimilar.  The  baseline  model  approach  gains  its  importance  by  establishing  a  basis 
for  comparison  between  the  provided  OSN  model  and  the  NPS  MANA  generated  model. 
This  is  accomplished  by  analyzing  the  outputs  of  each  of  the  models,  and  showing 
through  regression  that  the  models  are  very  similar,  to  the  point  of  interchangeability. 
The  advanced  model  approach  expands  upon  the  baseline  model  design  to  increase  the 
analyzed  space  to  try  and  answer  the  research  questions  presented.  Initially  this  focus  is 
to  stress  the  baseline  model  by  increasing  opposition  strength  and  behavior,  and 
observing  friendly  force  response.  The  following  portion  of  the  advanced  models  is  to 
provide  a  similar  increase  in  friendly  force  capabilities,  and  analyzing  whether  outcomes 
remain  similar  to  the  baseline  study,  or  if  other  operational  factors  become  more 
significant. 

B.  BASELINE  MODELS 

As  described  above,  the  baseline  study  is  as  attempt  to  generate  an  agent  based 
model  in  MANA  that  accurately  reflects  the  OSN  C++  agent  based  model.  This  model  is 
very  simple,  and  it  is  intended  to  be  this  way  in  order  to  initially  identify  important 
factors  of  the  OPV  prior  to  the  introduction  of  noise  variables. 

Why  use  MANA?  Nothing  in  this  thesis  is  intended  to  state  that  MANA  is  a 
superior  agent  based  modeling  software  to  others  that  are  currently  in  the  simulation 
market.  Its  use  is  specific  to  its  availability  and  its  compatibility  with  the  resources 
present  at  the  Naval  Postgraduate  School  and  the  SEED  Center.  That  being  established, 
MANA  does  provide  capabilities  that  were  easily  tailored  to  reflect  the  factors  present  in 
the  OSN  model.  Factors  like  range  and  Phit  probability  plots  and  agent  dependent  factors 
could  be  directly  implemented  in  MANA  with  little  effort.  In  addition  to  agent  factors, 
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replication  of  the  battle  space  in  MANA  was  straight  forward  using  parameters  provided 
from  the  OSN  scenario  descriptions. 

1.  OSN  ASUW  Scenario  Description 

The  OSN  model  is  a  battle  space  60  nautical  miles  wide  and  100  nautical  miles 
long.  A  simple  Cartesian  coordinate  system  is  used  to  establish  agent  location.  In  this 
scenario,  the  top  left  corner  of  the  battle  space  is  represented  by  the  (0,0)  point,  and  the 
bottom  right  represented  by  (60,100).  There  are  two  types  of  agents  in  this  battle  space, 
the  OP  Vs  and  the  SFACs.  There  are  a  total  of  12  agents  initially  on  each  model  run,  three 
OPVs  and  nine  SFACs.  There  is  one  additional  agent,  which  is  introduced  by  a  varying 
factor  dictating  the  presence  of  a  helicopter  in  support  of  the  OPVs. 


Figure  5.  OSN  Scenario  Map 
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Referencing  Figure  3,  the  OPVs  begin  model  runs  at  the  following  starting 
locations: 

•  OPV  1:  (5,25) 

•  OPV  2:  (25,25) 

•  OPV  3:  (45,25). 

OPVs,  once  generated,  begin  by  patrolling  an  established  route  in  defense  of  a  50 
nautical  mile  long  goal  line  located  at  (5,5)  and  extending  to  (55,5).  The  patrol  route  for 
each  OPV  is  listed  below: 

•  OPV  1  patrol  route:  (15,25),  (15,30),  (5,30),  (5,25) 

•  OPV  2  patrol  route:  (35,25),  (35,30),  (25,30),  (25,25) 

•  OPV  3  patrol  route:  (55,25),  (55,30),  (45,30),  (45,25). 

The  nine  SFACs  are  generated  in  a  50  nautical  mile  long  by  10  nautical  mile  wide 
homebox  with  coordinates: 

•  SFAC  Homebox:  (5,55),  (55,55),  (55,65),  (5,65). 

Their  placement  is  random,  and  once  generated,  they  proceed  directly  toward  the 
goal  line  with  the  shortest  route  possible.  They  do  not  deviate  from  this  course,  regardless 
of  the  presence  of  the  OPVs. 

The  helicopter  agent,  if  present,  is  generated  at  (30,50),  and  provides  detection 
and  classification  data  perfectly  to  the  OPVs. 

OPVs  patrol  their  prescribed  waypoint  list  at  a  set  speed  of  15  knots  in  the 
absence  of  detected  SFACs  or  SFAC  relayed  data  from  other  OPVs  or  the  helicopter.  If 
SFACs  are  detected,  a  target  list  is  generated,  and  SFAC  targets  are  distributed  to  the 
OPVs  by  a  decision  algorithm,  assigning  at  most  three  SFACs  to  each  OPV.  This 
algorithm  attempts  to  assign  SFAC  targets  to  the  closest  available  OPV.  Once  target  data 
is  received  by  the  OPV,  be  it  organic  or  inorganic,  the  OPV  accelerates  to  maximum 
speed  and  attempt  to  intercept  and  prosecute  the  SFAC.  OPV  weapon  priority  is  its  four 
available  missiles,  followed  by  a  naval  gun  if  equipped.  OPV  weapon  specifics  are 
detailed  below.  If  an  OPV  destroys  all  three  of  its  assigned  SFACs,  it  disengages  and 
proceeds  back  to  its  patrol  route.  These  successful  OPVs  attempt  to  kill  further  SFACs 
passing  within  range  if  they  still  have  ammunition  available,  but  they  do  not  maneuver  to 
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intercept.  Mission  success  for  the  OP  Vs  is  the  successful  defense  of  the  goal  line,  or  in 
other  words,  the  successful  elimination  of  all  nine  SFACs.  The  simulation  ends  if  OPVs 
achieve  mission  success,  or  if  a  SFAC  reaches  the  goal  line. 

a.  Model  Factors 

The  seven  factors  in  the  OSN  model  are  SFAC  Type,  OPV  Missile 
Type,  OPV  Gun  Present,  Helo  Present,  OPV  Max  Speed,  OPV  Mast 
Height,  OPV  Length  at  Waterline. 

•  SFAC  Type  -  A  binary  factor  of  Type  1  SFAC  or  Type  2  SFAC, 
which  adjusts  the  SFAC  maximum  speed  and  size.  Type  1  SFACs 
have  a  maximum  speed  of  25  knots  and  are  7  meters  tall.  Type  2 
SFACs  have  a  maximum  speed  of  36  knots  and  are  10  meters  tall. 

•  OPV  Missile  Type  -  A  binary  factor  of  missile  Type  1,  the  Marte 
missile  system,  or  Type  2,  the  Exocet  Missile  system. 
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Figure  6.  Marte  Missile  Phit  -  Range  Profile 
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Figure  7.  Exocet  Missile  P^t  -  Range  Profile 


•  OPV  Gun  Present  -  A  binary  factor  describing  the  presence  of  a 
naval  gun  on  the  OPV.  In  the  OSN  data,  this  factor  has  values  of 
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either  0  or  3,  with  three  representing  the  gun  is  present.  The  gun 
has  three  shots  of  ammo  with  the  below  Phit  distribution: 

Success 


Figure  8.  Gun  Phit  -  Range  Profile 


•  OPV  Max  Speed  -  A  continuous  factor,  with  a  range  of  22  knots  to 
40  knots  with  2  knot  increments,  describes  the  speed  an  OPV 
accelerates  to  when  it  detects  and  is  prosecuting  a  threat. 

•  OPV  Mast  Height  -  A  continuous  factor,  with  values  of  13,  16,  19, 
and  22  meters,  which  describes  the  height  of  the  equipped  radar 
system  on  the  OPV.  This  factor,  in  tandem  with  the  type  of  SFAC, 
determines  the  detection  and  classification  range  of  the  OPV.  For 
the  baseline  scenario,  the  detection  range  and  classification  range 
are  the  same. 

•  OPV  Length  at  Waterline  -  A  continuous  factor,  with  values  of  50, 
65,  and  80  meters,  which  influences  the  turn  rate  and  speed  of  the 
OPV. 

•  Helo  Present  -  A  binary  factor  of  0  or  1  which  describes  if  the 
helicopter  agent  is  present  in  the  scenario. 

b.  Factorial  Design 

To  explore  all  possible  data  points  in  the  design  space,  the  OSN  model 
used  a  full  factorial  design.  To  replicate  every  data  point  provided  by  the  OSN  design  and 
the  associated  data  could  be  computationally  expensive  given  that  there  are  seven  factors 
in  the  model,  some  of  which  are  not  binary.  Additionally  the  MOE  of  the  OSN  model  is 
the  percentage  of  time  the  OPV  successfully  defended  the  goal  line.  This  percentage  is  to 
the  first  decimal  place,  which  implies  each  data  point  relating  to  a  specific  set  of  model 
factor  values  was  run  with  at  least  a  thousand  replications.  Given  that  the  data  set 
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contains  roughly  1900  data  points,  the  full  factorial  would  require  almost  two  million 
model  runs. 

2.  MANA  Model  Differences 

While  many  of  the  specific  factors  and  their  associated  ranges  could  be  entered 
directly  into  MANA,  there  were  some  that  could  not,  or  required  modeling  finesse. 

a.  SFA  C  Horn  ebox 

While  the  dimensions  of  the  homebox  could  be  accurately  modeled  in  the 
MANA  environment,  the  behavior  of  agents  generated  in  the  box  created  issues  that  had 
to  be  addressed.  In  the  OSN  model,  SFACs  would  be  randomly  generated  in  the  homebox 
and  proceed  directly  north  to  the  goal  line  at  best  speed.  MANA’s  homebox  generates 
SFACs  in  the  same  manner,  but  agent  movement  is  directed  by  an  influence  system.  This 
influence  system  helps  to  prioritize  agent  behavior  in  a  model  run  as  the  modeler  dictates. 
For  the  SFACs,  that  behavior  is  set  at  100%  influence  to  proceed  to  the  first  waypoint, 
which  was  placed  in  the  same  location  as  the  goal  line  of  the  scenario.  The  issue  that 
arises  is  that  MANA  only  permits  a  series  of  waypoints  for  a  single  homebox,  not  a  goal 
line.  Therefore,  SFACs  generated  in  one  large  box  would  all  converge  on  a  single  point 
placed  on  the  goal  line,  which  was  the  first  waypoint.  This  behavior  is  undesirable  since 
it  would  impact  the  importance  of  the  OPV  patrolling  near  the  assigned  waypoint.  To 
overcome  this  problem,  the  single  large  homebox  as  described  in  the  scenario  was 
divided  into  2 1  smaller,  equally  sized  homeboxes  associated  with  the  SFAC  squad.  Each 
small  homebox  had  its  assigned  waypoint  directly  north  from  its  location,  creating  a  line 
of  waypoints.  This  series  of  2 1  waypoints  form  the  goal  line  in  the  MANA  environment. 
To  ensure  that  the  SFAC  agents  would  be  generated  in  the  same  manner  as  in  the  single 
larger  homebox,  a  size  proportionality  option  was  used.  This  feature  makes  the 
probability  of  having  a  SFAC  generated  in  a  particular  homebox  be  proportional  to  the 
box’  size.  Since  each  box  is  equal  in  size,  then  each  box  has  the  same  probability  to 
generate  every  SFAC  that  is  introduced  to  the  scenario.  Since  there  are  nine  SFAC 
agents,  not  every  box  would  generate  an  agent,  additionally  some  homeboxes  may 
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generate  more  than  one  agent.  This  allowed  for  the  same  behavior  as  a  single  large 
homebox  as  in  the  OSN  model. 

b.  Missile  Speed 

While  the  OSN  model  introduces  a  set  missile  velocity  of  300  meters  per 
second,  MANA  is  not  able  to  introduce  this  feature.  Since  missile  speed  is  present  in  the 
OSN  model,  a  delay  is  introduced  from  missile  launch  to  missile  hit  on  a  targeted  SFAC. 
This  delay  inhibits  the  OSN  OPVs  from  launching  another  missile  until  the  outcome  can 
be  determined  from  the  previous  launch.  If  the  missile  hits,  the  OPV  retargets  a  different 
SFAC  and  prosecutes  it.  If  it  missed,  the  OPV  fires  another  missile  and  repeat  the 
process.  In  MANA,  the  missile  launch  and  Phit  determination  occur  essentially 
instantaneously,  so  the  only  difference  between  the  two  models  is  the  delay  in  time 
associated  with  missile  launches.  This  delay  however  is  insignificant  in  simulation 
outcomes,  as  shown  in  the  analysis  portion  of  this  thesis. 

c.  OPV  Length  at  Waterline 

In  real  world  environments  and  ship  synthesis  considerations,  ship  length 
at  waterline  is  an  important  factor  since  it  impacts  speed,  turn  rate,  stability,  and  space 
considerations.  In  an  operational  modeling  environment  however,  that  impact  may  or 
may  not  be  significant.  In  the  OSN  model,  OPV  length  is  placed  as  a  varying  factor. 
MANA,  however,  cannot  replicate  this  feature  specifically,  yet  it  can  simulate  inertia  of 
individual  agents.  Values  for  this  inertia  factor  can  be  added  to  force  OPV  agents  to 
behave  like  naval  vessels,  but  there  does  not  exist  a  direct  way  to  compare  the  50,  65,  and 
80  meter  factor  set  points  for  the  OSN  model  to  specific  inertia  values  entered  into 
MANA.  This  inertia  value  is  set  to  15  in  the  model  runs,  since  it  is  stated  that  values 
above  10  begin  to  approach  ship  behaviors.  This  accounts  for  turn  and  speed  impacts  for 
the  OPVs  in  the  scenario.  For  the  space  considerations,  length  at  waterline  may  impact 
gun  and  missile  launcher  placement,  and  create  firing  arc  cutouts,  which  are  ranges  where 
the  weapons  cannot  fire  since  they  would  risk  striking  the  own  ship’s  superstructure.  It  is 
not  stated  if  cutouts  are  present  in  the  OSN  scenario,  therefore  the  length  at  waterline 
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factor  only  affects  maneuverability  of  the  OPVs.  This  impact  however  is  shown  to  be 
insignificant  in  the  analysis  section. 

d.  Contact  Prosecution 

As  mentioned  in  the  scenario  description,  contacts  detected  by  OPV  and 
helicopter  sensors  are  added  to  a  contact  list  which  is  in  turn  divided  amongst  the  OPVs 
for  prosecution.  MANA  does  not  behave  in  this  fashion  when  groups  of  agents  are 
targeted  by  other  groups  of  agents.  Agent  prosecution  is  handled  by  the  focusing  of 
movements  and  forces  to  the  geographical  center  of  gravity  (COG)  of  opponent  agents. 
For  the  MANA  baseline  model,  this  means  that  OPVs  maneuver  themselves  towards  a 
varying  point  near  the  center  of  all  detected  SFACs.  This  point  shifts  continuously, 
influenced  by  newly  detected,  lost,  or  destroyed  SFAC  agents.  A  problem  arises  when  a 
single  outlier  SFAC  is  generated  in  one  of  the  extreme  left  or  right  homeboxes,  and  the 
rest  are  grouped  on  the  opposite  side  of  the  environment.  OPVs  move  toward  the  cluster 
of  SFACs  since  the  COG  is  centered  near  them,  while  the  outlier  may  be  able  to  proceed 
to  the  goal  line  unmolested.  While  not  intended,  this  issue  is  pennitted  since  it  may  be  a 
tactic  actually  employed  by  real  life  agents  attempting  to  bypass  an  established  defended 
perimeter. 

When  addressing  the  established  SFAC  prosecution  list  from  the  OSN 
model,  MANA  agents  generally  engage  the  nearest  possible  target.  However,  MANA 
agents  can  be  provided  with  a  list  of  target  priorities  to  prosecute  in  order,  but  this  feature 
must  be  done  prior  to  model  runs,  and  cannot  be  randomly  generated.  Also,  the  modeler 
can  chose  to  allow  or  limit  agents  to  attack  their  nearest  threat  with  specific  weapons,  and 
reevaluate  their  targets  as  they  are  eliminated.  This  feature  was  used  in  the  MANA 
baseline  model,  so  OPVs  prosecute  the  nearest  detected  SFACs  first  with  missiles,  and 
then  work  out  towards  further  contacts. 

e.  OP  V  Max  Speed 

In  the  OSN  model  scenario  OPV  max  speed,  while  a  continuous  factor,  is 

modeled  with  integer  levels  between  22  to  40  knots  increasing  in  two  knot  increments. 

Speed  is  a  continuous  factor,  however,  and  it  is  feasible  that  insightful  information  can  be 
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drawn  from  speeds  allowable  within  the  OSN  established  levels.  Therefore,  the  DOE 
utilized  in  the  modeling  process  of  this  thesis  evaluates  speed  at  levels  over  the 
continuous  spectrum  between  the  high  and  low  setpoints. 

f.  OPV Mast  Height 

Similar  to  the  previous  argument  concerning  OPV  max  speed,  OPV  mast 
height  can  theoretically  exist  at  values  between  the  low  and  high  set  points  of  13  and  22 
meters  in  the  OSN  model.  This  being  the  case,  OPV  mast  height  was  modeled  as 
continuous  in  same  manner  as  OPV  max  speed. 

g.  Design  of  Experiments 

While  the  computational  resources  present  at  NPS  are  substantial,  the 
number  of  simulation  runs  to  replicate  the  full  factorial  is  excessive  given  that  there  are 
techniques  available  to  gain  insight  on  the  multidimensional  space  with  far  fewer  data 
points.  The  technique  which  was  employed  is  a  space  filling  design  that  drastically 
reduces  the  amount  of  required  computational  work  while  still  generating  samples 
spanning  the  range  of  the  full  factorial.  To  accomplish  this,  a  space  filling  design  of  two 
hundred  different  data  points  for  the  baseline  study  was  used.  This  design  covers  roughly 
twenty  percent  of  all  possible  data  points  and  each  individual  data  point  has  a  thousand 
replications  perfonned  to  generate  a  percent  success  of  OPV  goal  line  defense. 

3.  Limitations  and  Assumptions 

a.  Limitations 

1)  I  was  limited  in  my  ability  to  directly  replicate  the  OSN  model  since  it 
was  written  in  the  C++  computer  programming  language.  I  am  inexperienced  in  this 
computer  language,  and  I  do  not  have  sufficient  time  to  learn  it  and  rewrite  my  own  code 
to  produce  a  model  that  behaves  similarly  to  the  OSN  model. 

2)  I  was  limited  in  the  area  of  computer  replication  time  since  the  OSN 
design  of  experiments  explored  the  full  factorial  approach  to  data  collection.  To  perfonn 
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the  necessary  number  of  runs  to  achieve  the  full  factorial,  it  would  place  undue  strain  on 
highly  valuable  computational  resources  of  the  SEED  Center. 


b.  General  Assumptions 

1)  To  account  for  Limitation  #1,  I  have  assumed  that  MANA  is  an 
adequate  agent  base  modeling  software,  and  that  it  can  perfonn  in  similar  fashion  as  the 
OSN  model  if  I  utilize  it  proficiently. 

2)  To  account  for  Limitation  #2,  I  have  used  a  space  filling  design  that  is 
superior  to  the  full  factorial  in  terms  of  both  efficiency  and  coverage  of  the  factor  space. 


C.  ADVANCED  MODELS 

The  intent  of  the  advanced  scenarios  is  to  try  and  identify  any  change  or 
emergence  of  factor  importance  that  are  not  visible  in  the  baseline  models.  The  process  is 
a  two-step  approach,  in  that  initially,  the  SFACs  of  the  baseline  studies  are  enhanced  with 
behavioral  features  capable  in  MANA.  These  behavioral  enhancements  are  Avoidance 
and  Aggression.  For  Avoidance,  SFACs  attempt  to  evade  and  out-maneuver  the  OP  Vs 
while  attempting  to  reach  the  goal  line.  In  the  Aggression  scenarios,  SFACs  attack 
detected  OPVs  utilizing  suicide  weaponry  capable  of  eliminating  the  OPVs  from  the 
battlefield.  Additionally,  a  third  scenario  consisting  of  a  combination  of  Avoidance  and 
Aggression  SFACs  are  implemented  to  stress  the  OPVs.  Each  scenario  is  run  with  the 
OPVs  generated  during  the  baseline  study  to  evaluate  their  ability  to  counter  a  more 
robust  threat.  Once  these  three  models  are  run,  another  set  of  analyses  are  conducted 
utilizing  enhanced  OPVs  with  capabilities  established  in  Abel  (2009),  Ozdemir  (2009), 
and  Cobian  (2002).  These  improved  OPVs  are  placed  in  the  same  scenarios  described 
previously  to  see  if  any  emergent  behavior  or  dissimilar  outcomes  appear.  For  each  of  the 
advanced  scenarios,  the  number  of  SFACs  has  been  increased  from  nine  to  eighteen 
agents  to  help  focus  the  performance  of  the  OPVs  on  their  associated  weapons  and 
capabilities. 
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1. 


SFAC  Avoidance  vs.  Baseline  Scenario 


As  mentioned  in  the  baseline  description  of  the  scenarios,  MANA  utilizes  a 
system  of  influences  to  help  direct  agent  motion  in  the  simulated  environment.  In  the 
baseline  scenario,  the  SFAC  agent’s  only  priority  was  to  reach  the  goal  line  using  the 
shortest  route  from  their  spawn  location.  This  influence  has  been  altered  by  introducing  a 
detection  capability  in  the  SFAC  agents  comparable  to  the  OPVs’.  With  this  capability, 
SFACs  both  see  and  identify  approaching  OPVs  as  a  threat  and  attempt  to  avoid  them, 
while  at  the  same  time,  attempting  to  reach  the  goal  line. 

Given  the  small  nature  of  the  SFAC  in  relation  to  sea  state  inherent  to  open  ocean 
conditions,  a  stealth  feature  available  in  MANA  has  been  activated  for  the  small  boat 
agents.  This  stealth  set-point  is  established  at  70%  effectiveness.  What  this  means  is  that 
for  any  individual  sweep  of  radar,  there  is  a  30%  chance  to  detect  the  SFAC.  While  a 
disadvantage  for  a  single  scanner,  the  tactical  sharing  of  target  infonnation  between 
OPVs  allows  for  areas  of  coverage  being  swept  by  multiple  radars,  thus  increasing 
probability  of  detection.  The  resulting  effect  is  that  when  SFACs  are  more  distant  to 
OPVs  and  outside  multiple  radar  sweep  areas,  they  are  harder  to  see.  However,  detection 
improves  as  OPV  and  SFAC  separation  decreases.  This  decrease  in  visibility  as  a  small 
object  gets  further  from  an  observer  is  more  realistic  in  nature,  and  coincides  with 
operational  experience. 

Aside  from  the  above  modifications  to  the  MANA  model,  no  other  changes  have 
been  made  to  the  baseline  model.  This  effort  is  to  try  and  identify  if  factor  significance  is 
altered  by  SFAC  motion  and  avoidance. 

2.  SFAC  Aggressive  vs.  Baseline  Scenario 

Along  with  the  detection  and  stealth  capabilities  applied  to  the  SFAC  agents  in 
the  Avoidance  scenario,  SFACs  in  the  Aggressive  scenario  were  enhanced  with  suicide 
weaponry  capable  of  destroying  the  patrolling  OPVs.  The  influence  system  for  this 
scenario  has  been  reversed  when  SFACs  detect  OPVs.  Instead  of  fleeing  from 
approaching  OPVs,  SFACs  close  and  attempt  to  collide  with  the  OPVs  to  try  and  destroy 
them.  This  tactic  has  some  historical  credence  since  it  was  utilized  during  the  Sri  Lankan 
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Civil  War  by  the  Tamil  Sea  Tigers  in  order  to  cripple  and  demoralize  Sri  Lankan  naval 
forces. 


3.  SFAC  Combination  vs.  Baseline  Scenario 

The  Combination  scenario  is  a  blend  of  both  the  Avoidance  and  Aggressive 
scenarios.  Half  of  the  SFAC  agents  follow  the  Avoidance  strategy,  while  the  other  half 
follow  the  Aggressive  mindset.  The  goal  of  this  scenario  is  to  identify  if  Aggressive 
behaving  SFACs  influence  the  OP  Vs’  capability  in  intercepting  them,  and  the  Avoidance 
SFACs. 

4.  OPV  Enhancements 

The  characteristics  of  the  OP  Vs  have  been  altered  for  the  next  three  scenario  runs 
in  an  effort  to  introduce  more  realism  to  the  modeling  environment.  While  the 
effectiveness  of  the  equipped  Marte  and  Exocet  missile  systems  has  gone  unaltered,  the 
gun  mount  has  been  significantly  changed  to  better  reflect  actual  naval  gun 
characteristics.  Additionally,  naval  ships  currently  being  designed  and  produced  have 
begun  the  employment  of  new  weapons  systems  aimed  at  countering  the  small  boat 
swarm  threat.  The  Griffin  missile  system  is  one  such  weapon,  given  its  employment  on 
the  U.S.  Navy’s  new  LCS  ships.  While  the  weapon  system  is  still  undergoing  testing,  the 
addition  of  this  weapon  acknowledges  the  significance  of  the  small  boat  threat.  Finally,  a 
behavioral  enhancement  for  OP  Vs  has  been  implemented  to  help  maintain  adequate 
separation  distances  between  approaching  SFAC  agents.  This  behavior  is  intended  to 
parallel  current  close  approach  and  collision  avoidance  situations  practiced  by  warships 
at  sea  today. 


a.  Naval  Gun 

In  the  baseline  scenario,  prescribed  gun  Phit  effectiveness  was  0.5  for 
targets  within  one  nautical  mile,  decreasing  linearly  to  0.4  for  contacts  within  four 
nautical  miles.  This  weapon  effectiveness  is  unrealistic  when  referencing  historical 
records  of  naval  gunfire  in  wartime  environments.  As  an  example,  of  the  531  U.S.  PT 
boats  used  in  WWII  by  the  U.S.  Navy,  only  seven  of  the  total  99  lost  were  due  to  enemy 
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naval  gunfire  (Bulkley,  1962).  This  equates  to  a  Phit  of  0.0132,  and  this  value  is  non¬ 
range  specific,  which  means  the  seven  PT  boats  destroyed  by  naval  gunfire  could  have 
been  destroyed  at  far  range  or  when  close  aboard  their  adversaries.  This  low  Phit  also 
correlates  with  gunfire  effectiveness  seen  at  the  Battle  of  the  Falklands  on  December  8, 
1914  where  British  estimates  put  gun  hit  accuracy  near  1%.  This  engagement  took  place 
between  large,  slow  moving  warships  equipped  with  12”  guns,  not  small  and  agile  fast 
boats  moving  in  excess  of  30  knots.  While  it  can  be  argued  that  naval  fire-control 
technology  is  far  more  advanced  than  WWI  and  WWII  systems,  the  speed  and  agility  of 
modem  day  SFACs  are  significantly  higher  than  comparable  boats  of  that  era.  Ultimately, 
professional  experience  and  historical  records  help  to  dictate  probability  of  hit 
characteristics  far  lower  than  those  provided  by  the  OSN  baseline  model. 

Developed  in  1964  by  the  Italian  weapon  manufacturer,  Oto  Melara,  the 
76mm  Otobreda  naval  gun  is  one  of  the  most  successful  and  widely  used  naval  guns  in 
the  world  today. 


Figure  9.  76mm/62  Caliber  Otobreda  Naval  Gun 


Used  in  53  navies  worldwide,  this  gun  is  capable  of  firing  271b  projectiles 

at  a  rate  of  85  rounds  per  minute  with  a  maximum  range  of  20  kilometers.  Mounted  on 

many  frigate  class  ships,  such  as  the  U.S.  Navy’s  Oliver  Hazard  Perry  class,  this  gun  is 

31 


also  compact  enough  to  be  mounted  on  smaller  corvette  and  OPV  class  ships.  Given  its 
country  of  origin,  compatibility,  and  extent  of  proliferation,  the  76mm  Otobreda  has  been 
chosen  to  be  modeled  in  the  enhanced  OPV  model  runs. 
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1. 


Model  Specifics 


Status  of  Weapons:1  2  3  4  5  6 
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Figure  10.  Screenshot  of  MANA  Naval  Gun  Characteristics 


The  characteristics  of  the  naval  gun  equipped  on  OP  Vs  in  the  enhanced 
model  runs  are  detailed  in  Figure  12.  While  the  probability  of  hit  at  or  near  point  blank 
range  is  still  a  modest  50%,  the  decrease  in  gun  hit  effectiveness  falls  off  exponentially  to 
1%  at  four  nautical  miles.  While  it  may  be  argued  that  even  this  level  of  effectiveness  is 
still  too  optimistic  given  the  nature  of  small  boat  engagement,  without  the  availability  of 
actual  naval  gunfire  effectiveness  data  in  small  boat  attacks,  this  assumption  seems  much 
more  reasonable  that  the  previously  prescribed  gun  hit  profiles  in  the  baseline  models. 

Additionally,  the  ammunition  considerations  of  the  baseline  model  have 
been  altered  to  a  more  realistic  value.  In  the  prior  models,  OPVs  were  allowed  three 
engagements  with  approaching  SFACs  with  the  equipped  gun.  This  does  not  coincide 
with  typical  ammunition  load-outs  of  warships  at  sea.  The  76mm  Otobreda  naval  gun,  for 
instance,  contains  one  full  minute’s  worth  of  ammunition  in  the  gun  magazine,  or  in  other 
words,  85  shells.  This  amount  of  ammunition  is  assumed  sufficient  for  the  modeled 
engagements  since  there  would  not  be  adequate  time  available  to  reload  an  expended 
magazine  prior  to  the  completion  of  a  simulated  run. 
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b.  Griffin  Missile  System 

Given  the  recent  investment  of  advanced  weapons  to  combat  the  small 
boat  swann  threat,  the  U.S.  Navy’s  LCS  program  gives  credence  to  the  severity  of  the 
SFAC  threat.  Partly  as  a  counter  to  published  Iranian  naval  warfare  doctrine,  and  its 
reliance  on  small  boat  swarms,  the  U.S.  has  contracted  the  weapons  manufacturing 
company,  Raytheon,  to  help  combat  the  growing  problem.  Their  answer  is  the  Griffin 
Missile  System. 


Figure  1 1 .  Raytheon  Griffin  Missile  Test  Launch 

Originating  from  Ratheon’s  Rolling  Airframe  Missile  (RAM),  the  Griffin 
is  a  smaller,  less  expensive  weapon  designed  to  engage  close  range  surface  threats.  With 
engagement  ranges  of  only  a  few  miles,  the  Griffin  is  slotted  as  a  defensive  weapon, 
employed  in  tandem  with  other  shipboard  defensive  systems  such  as  the  CIWS,  crew 
served  weapons,  and  the  naval  gun.  On  June  15,  2012,  the  U.S.  Navy  engaged  three 
SFAC  targets  on  a  sea-based  platfonn  with  three  laser  guided  Griffin  missiles,  each 
scoring  direct  hits  (Paris,  2012).  While  the  specifics  of  the  Griffin  hit  and  kill 
probabilities  are  currently  unknown,  previous  weapon  systems  such  as  the  Hellfire  and 
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RAM  missiles  have  proven  to  possess  very  high  Ph«  probabilities.  The  specifics  of  the 
MANA  modeled  Griffin  missile  system  is  shown  in  Figure  14  below. 


Status  of  Weapons:!  2  3  4  5  6 


Giiffin 


Cl.iss 


©Pii  OSec 


Weapon: 


•j  ® 

3  a 


0 


Master 

Enable 


Weapon  Model  ©Simple  f) Advanced 


0/.OC/1  Parameter  Values  to  Default  State  file  Mode  Target  Kinetic  Eneigy  Agent  SA 
0  Walls  and  Hills  Block  File  Shots  Ammo 


m 


Reload  Time 


m 

(▼) 


(seconds) 


0 


Enable  In 
This  State 


Copy  Wpn 
State  Values 


Paste  Wpn 
State  Values 


Shots  pel  Second 


too 


100 


Aiiiioiii  Penetiationo 


I _ Si 


M  """ 


Range.  R  (meties) 

Hit  Rate  per 
Discharge  (r<=R) 


□ 


Fire  on  Closest 
Targets  First 


□  interpolate  Within  Subranges  on  SSKP  Table 


Shot  Radius 


(▼) 


Figure  12.  Screenshot  of  MANA  Griffin  Missile  Characteristics 


While  the  number  of  missiles  loaded  on  naval  Griffin  launchers  is 
unknown,  since  the  missile  heralds  its  origins  from  the  RAM  system  employed  on  naval 
warships  for  air  defense  it  is  assumed  that  the  Griffin  holds  a  number  of  missiles  per 
launcher  comparable  to  the  RAM.  This  ultimately  creates  a  very  effective,  short  range 
defensive  weapon  system  to  combat  the  small  boat  swann  threat. 

c.  Risk  of  Collision  Situations 

U.S.  naval  warships  operating  on  the  oceans  today  do  so  by  adhering  to  a 
strict  set  of  instructions  put  forth  by  their  commanding  officers  (CO).  These  instructions 
range  from  permission  items  to  subordinate  officers  onboard,  to  rules  and  expectations  of 
key  watch-stations  performing  their  routine  duties.  One  such  document  readily  known  to 
most  surface  warfare  officers  (SWO)  is  the  CO’s  Standing  Orders  (CSO).  The  CSO  is 
evaluated  and  produced  by  every  CO  upon  their  arrival  to  a  new  command,  and  it  dictates 
how  watch-teams  conduct  their  watches  at  key  locations,  such  as  the  Bridge,  Combat 
Information  Center  (CIC),  and  Central  Control  Station  (CCS).  For  the  bridge  team,  safe 
maneuvering  of  the  ship  is  of  principle  importance,  and  the  most  serious  of  situations  for 

the  bridge  team  to  face  is  the  Risk  of  Collision  (ROC)  situation.  Collision  at  sea  is  a 
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serious  problem,  not  only  because  it’s  an  embarrassment  for  the  CO.  It  damages  the  ship, 
injures  crew,  and  possibly  impacts  the  environment.  Given  its  severity  many  COs  detail 
their  expectations  to  the  bridge  in  regards  to  ROC,  usually  involving  CO  notification, 
statements  of  intention  to  the  involved  vessels,  and  avoidance.  The  avoidance  portion  of 
ROC  is  important  for  this  study  since  in  the  previous  Aggressive  scenario,  OP  Vs  would 
not  attempt  to  avoid  approaching  SFACs  laden  with  explosives.  This  resulted  in  the  loss 
of  OP  Vs  which  may  have  been  able  to  avoid  and  defend  themselves  from  SFAC  attack, 
and  thus  continue  to  defend  the  objective.  From  operation  experience,  avoiding  ROC  can 
take  place  at  any  range,  but  generally  ship  course  alterations  become  more  drastic  as 
distance  and  time  to  collision  decreases.  In  the  enhanced  OPV  MANA  runs,  an  extreme 
ROC  situation  is  deemed  to  exist  when  SFACs  encroach  within  two  nautical  miles.  Once 
this  distance  is  violated  by  attacking  SFACs,  OPVs  attempt  to  avoid  and  steam  away 
from  threats  as  quickly  as  possible.  At  this  point,  OPVs  continue  to  eliminate  the 
pursuing  threat  as  best  as  possible,  and  only  return  to  previous  operations  when  all 
SFACs  are  removed  from  the  two  nautical  mile  radius. 

D.  MODEL  REPLICATION  DISTRIBUTION 

As  previously  mentioned,  the  baseline  MANA  model  consisted  of  200  distinct 
data  points,  with  one  thousand  runs  per  point.  This  resulted  in  200  rows  of  averaged  data, 
producing  a  MOE  comparable  to  the  OSN  baseline  data.  For  the  advanced  model  runs, 
the  number  of  replications  is  significantly  reduced,  since  it  is  the  effort  of  the  baseline 
model  to  establish  interchangeability  between  the  OSN  and  MANA  baseline  models. 
With  this  interchangeability  established,  it  can  be  presumed  that  if  the  capabilities  present 
in  the  advanced  MANA  models  were  available  in  the  OSN  modeling  environment,  that  it 
would  produce  similar  results.  Below  is  the  distribution  of  model  runs  for  the  advanced 
MANA  models. 

•  Advanced  Model  (Avoidance  Scenario)  -  200  data  points,  with  50 
replications  per  data  point 

•  Advanced  Model  (Aggressive  Scenario)  -  200  data  points,  with  50 
replications  per  data  point 

•  Advanced  Model  (Combination  Scenario)  -  200  data  points,  with  50 
replications  per  data  point 
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•  Enhanced  OPV  Model  (Avoidance  Scenario)  -  200  data  points,  with  50 
replications  per  data  point 

•  Enhanced  OPV  Model  (Aggressive  Scenario)  -  200  data  points,  with  50 
replications  per  data  point 

•  Enhanced  OPV  Model  (Combination  Scenario)  -  200  data  points,  with  50 
replications  per  data  point 
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IV.  ANALYSIS 


A.  INTRODUCTION 

The  prior  three  chapters  have  shed  light  onto  the  initial  situation,  historical 
influence,  and  steps  taken  to  develop  the  models  necessary  to  answer  the  research 
questions.  This  chapter  shows  the  analysis  techniques  and  tools  utilized  to  scrutinize  the 
data  for  insightful  information.  After  this  overview,  an  in-depth  analysis  of  the  produced 
data  was  conducted  between  the  baseline  and  advanced  model  datasets. 

Initial  analysis  efforts  focus  on  model  interpretation  and  interchangeability 
between  the  OSN  and  MANA  models.  Additionally,  this  analysis  results  in  the 
detennination  of  factor  importance  in  the  baseline  scenarios.  These  results  are  important 
because  they  establish  the  basis  which  the  advanced  model  factors  are  compared  against. 

The  analysis  of  the  advanced  models  consists  of  two  phases,  first  being  focused 
on  the  impact  of  SFAC  improvements  and  behaviors  on  the  baseline  modeled  OPVs.  It  is 
the  goal  of  this  portion  of  the  study  to  understand  the  impact  of  the  increased  capabilities 
allowable  in  MANA.  Secondly,  this  work  addresses  the  enhancement  of  the  OPVs,  and 
how  the  introduction  of  more  realistic  capabilities  of  a  warship  compare  to  the  improved 
SFAC  problem.  It  is  from  this  point  that  the  following  conclusions  and  results  are  based 
upon. 

B.  ANALYSIS  TOOLS  AND  METHODS 

Before  any  useful  information  can  be  pulled  from  the  raw  data,  an  inspection 
needs  to  be  conducted  to  identify  any  errors  or  data  corruption.  After  this  inspection,  the 
data  then  needs  to  be  molded  into  a  fonn  that  can  be  fed  into  an  analysis  tool.  The 
analysis  tool  utilized  in  this  analysis  is  JMP  9  Pro  due  to  its  insightful  visual  outputs  as 
well  as  its  ease  of  use  in  drawing  out  constructive  output. 
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1.  Data  Inspection 

a.  Baseline  Model  Data 

As  mentioned  in  the  first  chapter,  the  OSN  data  set  consists  of  1921  rows 
of  data  with  a  response  as  a  percentage  chance  of  successful  defense,  taken  to  the  first 
decimal  place.  The  OSN  model  explores  the  full  factorial,  which  means  every  possible 
combination  of  factor  settings  has  been  investigated  thus  creating  a  complete 
representation  of  the  entire  multidimensional  space.  A  sample  of  the  OSN  data  is  shown 
in  Table  2. 
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Table  2.  Portion  of  OSN  Dataset 


As  mentioned  in  the  Methodology  chapter,  Table  2  shows  all  seven  factors 
being  analyzed  and  a  portion  of  the  factor  levels. 

The  data  generated  from  the  baseline  MANA  model  consisted  of  two 
hundred  data  points,  utilizing  a  space  filling  design  as  detailed  earlier.  Each  data  point 
was  simulated  one  thousand  times  to  generate  a  mean  response  comparable  to  the  OSN 
data  set.  A  sample  of  the  averaged  baseline  MANA  data  is  shown  in  Table  3. 
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Table  3.  Portion  of  MAN  A  Baseline  Dataset 


As  explained  in  the  Methodology  portion,  the  baseline  data  generated 
from  the  MANA  model  does  not  contain  the  Length  at  Waterline  (LWL)  factor 
that  is  present  in  the  OSN  model.  The  data  are  also  averaged  to  generate  the 
percentage  MOE  shown  in  the  seventh  column.  Each  row  represents  one  thousand 
model  runs  with  the  shown  factor  set  points,  and  the  resulting  averaged  response. 

b.  Advanced  Model  Data 

Each  of  the  advanced  scenario  datasets  consists  of  200  design  points,  with 
every  point  run  50  times  with  prescribed  values  for  each  factor,  resulting  in  10,000 
simulation  runs.  Since  there  are  six  advanced  model  scenarios  being  evaluated,  the  end 
product  for  the  advanced  data  is  60,000  rows.  Fortunately,  given  the  available 
computational  power  present  in  the  SEED  Center  at  NPS,  the  generation  of  the  data  took 
an  acceptable  amount  time. 

As  stated  previously,  the  advanced  scenarios  are  intended  to  help  us 
understand  and  show  the  changes  in  factor  importance  when  placed  in  differing 
environments.  Keeping  this  in  mind,  the  MOP  of  OPV  systems  is  evaluated  to  a  higher 
degree  than  the  MOE  of  the  baseline  models.  As  an  example,  taking  a  look  at  OPV 
survival  rates  now  becomes  an  important  topic  since  it  was  impossible  to  lose  an  OPV 
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during  the  baseline  study.  Additionally,  given  the  changes  to  the  OPV  gun  in  the 
enhanced  OPV  scenarios,  its  MOP  is  expected  to  differ  greatly  from  the  baseline  OPV 
scenarios. 

2.  Analysis  Tool 
a.  JMP 

Originally  named  John’s  Macintosh  Project  (JMP)  prior  to  its  release,  JMP 
is  a  statistical  analysis  package  that  was  developed  for  Macintosh  machines  in  1993  and 
later  for  Windows  and  Linux  computers.  Named  after  the  developer,  John  Stall,  JMP 
allows  for  extensive  statistical  analysis  with  graphical  display  and  interactivity.  With  its 
ease  of  use  and  compatibility  with  a  variety  of  data  formats,  JMP  is  a  widely  used  tool  for 
conducting  simple  and  advanced  data  analysis.  The  edition  utilized  in  this  study  is  JMP  9 
Pro  (SAS  Institute  Inc.,  2012). 

3.  Statistical  vs.  Practical  Significance 

During  the  course  of  the  analysis,  it  is  important  to  remember  the  difference 
between  statistical  and  practical  significance.  Statistical  significance  is  a  function  of  the 
sample  size,  and  indicates  that  the  differences  in  the  mean  are  not  a  result  of  chance  and 
rejection  of  the  null  hypothesis  is  true.  However,  it  does  not  indicate  importance  or  the 
severity  of  these  differences.  This  is  shown  by  practical  significance,  which  identifies  if  a 
difference  in  the  response  has  real-world  implications.  An  example  being  that  a 
difference  in  mast  height  by  1  inch  on  an  OPV  may  be  statistically  significant  for  SFAC 
detection  purposes,  but  is  the  cost  associated  with  designing  and  raising  that  mast 
practical. 

C.  INTERCHANGEABILITY 

Prior  to  the  implementation  of  the  advanced  modeling  effort,  the  comparison  of 
the  baseline  models  needs  to  be  conducted.  The  reason  behind  this  is  before  advanced 
features  available  in  MANA  can  be  utilized,  it  has  to  be  shown  that  both  the  OSN  and 
baseline  MANA  models  behave  in  very  similar  manners.  This  similarity  in  behaviors 
needs  to  be  to  such  a  level  that  they  could  be  used  interchangeably.  The  establishment  of 
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interchangeability  would  allow  us  to  state  that  if  the  OSN  model  had  all  the  capabilities 
utilized  in  the  advanced  MANA  scenarios,  that  it  should  produce  similar  results. 

Regression  analysis  and  other  techniques  are  utilized  to  establish 
interchangeability,  and  the  analysis  is  shown  below. 


1.  OSN  Baseline  Model  Analysis 

The  first  step  down  the  road  to  interchangeability  is  the  establishment  of  an 
objective  to  work  towards,  and  for  this  research  effort,  it  was  the  replication  of  the  work 
performed  by  the  OSN  staff.  Data  produced  from  their  efforts  was  made  available  to  the 
OR  students  participating  in  the  ASNET/PRONT  NICOP  Project.  The  focus  of  the  effort 
was  not  to  establish  validation,  but  verification.  This  approach  helps  to  confirm  that  the 
previous  study  was  not  influenced  by  unidentified  factors  or  errors  which  may  have 
impacted  the  outcome.  To  verify  the  OSN  model,  its  output  had  to  be  presented,  with  its 
MOE  and  MOP  of  involved  factors. 


zd  Summary  of  Fit 
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Figure  13.  OSN  Model  Regression 
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The  regression  plot  and  statistical  output  in  Figure  15  lays  the  initial  baseline  for 
the  following  NPS  effort  in  regards  to  the  ASUW  scenario.  The  very  high  R-squared 
states  that  most  of  the  variance  in  the  data  is  accounted  for  by  the  model.  Looking  at  the 
statistical  output  gives  credence  to  the  assumption  that  this  model  is  a  very  good 
reflection  of  the  real  data,  but  a  qualitative  approach  to  looking  at  the  Actual  vs. 
Predicted  Plot  begins  to  generate  questions. 


zd  Actual  by  Predicted  Plot 


Figure  14.  OSN  Model  Actual  by  Predicted  Plot 

The  distinct  groupings  present  in  the  plot  infer  that  there  may  be  particular  factors 
which  drastically  influence  the  response.  To  try  and  identify  these  relationships,  partition 
trees,  sorted  parameter  estimates,  or  factor  prediction  profilers  are  a  few  of  the  tools 
available  in  JMP. 
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a  Sorted  Parameter  Estimates 
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<.0001* 

SSM[1] 

-7.694375 

0.064766 

-118.8 

<.0001* 

HELO[0] 

-5.30791  7 

0.064766 

-81.95 

<.0001* 

SFAC[1] 

3.303541  7 

0.064766 

51.01 

1 

<.0001* 

Max  Speed 

0.5452525 

0.011274 

48.36 

J  ! 

<.0001* 

GUN[0]*HELO[0] 

-2.922708 

0.064766 

-45.13 

!  ! 

<.0001* 

HELO[OfSFAC[1] 

-2.045208 

0.064766 

-31.58 

<.0001* 

GUN[0]*(Max  Speed-31) 

-0.344855 

0.011274 

-30.59 

<.0001* 

GUN[0]*SSM[1] 

-1.551667 

0.064766 

-23.96 

<.0001* 

GUN[0]*SFAC[1] 

-1.516667 

0  064766 

-23.42 

<.0001* 

HELO[0]*(Max  Speed-31) 

-0.244495 

0.011274 

-21.69 

<.0001* 

HELO[0]*SSM[1] 

1.1352083 

0.064766 

17.53 

<.0001* 

MastH 

0.2646389 

0.01931 

13.71 

<.0001* 

(Max  Speed-31)*(Max  Speed-31) 

-0.022029 

0.002228 

-9.89 

<.0001* 

(Max  Speed-31  )*SFAC[1] 

-0.107557 

0.011274 

-9.54 

<.0001* 

HELO[0]*(MastH-1 7.5) 

0.1211944 

0.01931 

6.28 

<.0001* 

(Max  Speed-31  )*SSM[1] 

0.0609533 

0.011274 

5.41 

<.0001* 

SFAC[1]*SSM[1] 

0.2441667 

0.064766 

3.77 

0.0002* 

(MastH-1 7.5)*SSM[1] 

-0.0675 

0.01931 

-3.50 

0.0005* 

(Max  Speed-31)*(MastH-1 7.5) 

-0.00942 

0.003361 

-2.80 

0.0051* 

Figure  15.  OSN  Model  Sorted  Parameter  Estimates 


Right  off  the  bat,  the  naval  gun  factor  stands  out  from  the  sorted  parameter 
estimates  plot  of  the  OSN  data,  so  the  distinct  groupings  present  in  the  regression  plot 
may  have  some  reliance  on  whether  or  not  the  gun  is  present  in  a  simulation  run. 


Figure  16.  OSN  Model  Prediction  Profiler  Plot 


Looking  at  the  prediction  profiler  plot  of  the  OSN  data  yields  a  similar 
conclusion;  the  gun  factor  has  the  largest  impact  on  the  response.  As  a  reminder,  the  gun 
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factor  is  binary:  a  value  of  0  signifies  the  gun  is  absent  from  the  OPV  simulation,  while  a 
value  of  3  represents  the  presence  of  the  gun  that  has  three  shots  available  to  prosecute 
the  SFACs. 


A  -  Partition  for  MOE 


Figure  17.  OSN  Model  Partition  Tree  Plot  (1st  Split) 


From  the  partition  tree  projection  of  the  OSN  model  data,  with  the  very  first  split, 
the  gun  factor  establishes  itself  as  the  key  deciding  factor  in  regards  to  the  response. 
Additionally,  with  just  one  factor,  79.6%  of  all  variation  in  the  data  is  being  accounted 
for  according  to  the  R-squared  value  in  Figure  19. 

With  each  of  the  three  previous  analysis  tools  pointing  toward  the  naval  gun 
factor  as  being  the  most  significant,  trying  to  understand  how  this  factor  is  impacting  the 
regression  model  in  the  creation  of  the  distinct  groupings  is  of  importance.  To  do  this, 
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JMP’s  user  interface  (UI)  can  be  employed  to  select  all  data  points  associated  with  the 
presence  of  the  naval  gun  factor,  or,  its  absence. 


Figure  18.  OSN  Model  Actual  by  Predicted  Plot  (Gun  absent  selection) 

With  the  specific  selection  of  all  rows  of  data  in  the  OSN  model  output  without 
the  presence  of  a  naval  gun,  it  can  clearly  be  seen  that  both  of  the  lower  groupings  and 
the  later  portion  of  the  larger  grouping  are  included.  This  implies,  along  with  the  previous 
analysis  results,  that  the  naval  gun  is  the  most  significant  factor  impacting  model 
response.  Yet  it  does  not  explain  the  full  picture  in  regards  to  the  groupings  since  both  of 
the  smaller  groups  lie  within  the  data  set  missing  the  gun  factor,  so  additional  factors  may 
be  impacting  the  creation  of  the  lower  performance  groupings. 

The  other  realized  piece  of  information  from  Figure  20  is  that  regardless  of  the 
rest  of  the  OPV  armaments  or  performance  capabilities,  the  inclusion  of  the  naval  gun 
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produces  results  in  the  upper  half  of  all  model  simulation  runs.  While  some  overlap 
between  the  gun  being  present  and  not  being  present  exists,  the  vast  majority  of  available 
data  existing  above  the  Mean  Response  of  52.3  involves  the  inclusion  of  the  gun  factor. 

Referencing  Figure  17  again,  the  Surface  to  Surface  Missile  (SSM)  factor,  or 
OPV  Missile  Type,  is  the  next  factor  with  the  highest  T-ratio,  meaning  that  it  is  both 
statistically  significant,  and  strongly  defeats  the  null  hypothesis.  Checking  the  partition 
tree  further  shows  its  importance  to  the  OSN  modeling  analysis. 
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Figure  19.  OSN  Model  Partition  Tree  Plot  (2nd  Split) 
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Confirming  the  result  shown  in  Figure  17  in  regards  to  SSM  importance,  Figure 
2 1  shows  that  which  OPV  missile  type  is  equipped  has  the  second  largest  impact  on  the 
response  following  the  naval  gun.  The  increase  in  R-Squared  also  shows  its  importance 
given  that  the  SSM  factor  raised  R-Squared  by  almost  6%  from  the  previous  79.6% 
established  by  the  gun  factor.  With  this  result  in  mind,  the  picture  concerning  the 
regression  plot  groupings  begins  to  clear,  and  a  relation  between  the  two  factors  may  be 
the  reason  behind  the  groupings. 
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Figure  20.  OSN  Model  Actual  by  Predicted  Plot  (Gun  Absent  &  SSM  1  Equipped) 


In  Figure  20,  the  data  highlighted  identifies  OPVs  lacking  a  naval  gun  and  having 
SSM  1  equipped.  The  highlighted  data  accounts  for  all  data  points  located  in  the  lowest 
grouping,  as  well  as  a  portion  of  the  next  higher  grouping.  This  result  passes  the  common 
sense  test  since  a  warship  equipped  with  the  poorest  weapon  system,  and  lacking  a  gun  to 
prosecute  a  threat  would  be  expected  to  perform  below  par.  For  the  other  portion  of  data 
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in  the  second  grouping,  looking  at  Figure  17  again  shows  that  the  presence  of  the 
helicopter  is  the  next  most  important  factor. 


zi  ■*  Partition  for  MOE 


Figure  2 1 .  OSN  Model  Partition  Tree  Plot  (Splits  1-4) 
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Figure  23  reaffirms  the  previous  statement  that  a  helicopter  feeding  data  to  the 
patrolling  OP  Vs  is  the  next  most  significant  factor.  The  infonnation  distribution 
capability  provided  by  the  helicopter  in  the  simulations  drastically  increases  the  OPV 
detection  effectiveness,  and  allows  for  missile  engagements  on  the  SFACs  to  be  taken  at 
significantly  longer  ranges. 


Figure  22.  OSN  Model  Actual  by  Predicted  Plot  (Gun  &  Helicopter  Absent,  SSM  1) 


To  confirm  the  result  from  Figure  23,  Figure  24  shows  that  exclusion  of  the  naval 
gun  and  the  helicopter  providing  tactical  data  creates  a  worst  case  situation  for  the  OP  Vs 
with  the  SSM  1  missile  system  is  equipped,  regardless  of  speed  of  the  OPV,  LWL,  or 
which  type  of  threat  they  are  facing. 

a.  Length  at  Waterline 

As  previously  mentioned,  the  MANA  baseline  model  was  unable  to 
account  for  OPV  length  at  waterline,  which  was  included  in  the  OSN  modeling  effort. 
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Referencing  Figure  17  shows  that  neither  the  LWL  factor,  nor  any  interactions  with  this 
factor  were  deemed  statistically  significant  to  the  OSN  model  regression.  Ultimately  this 
means  that  the  LWL  factor  can  be  removed  from  consideration  because  it  does  not  impact 
model  performance.  This  does  not  however  mean  that  LWL  is  not  important  for  ship 
synthesis  considerations,  because  the  length  of  a  naval  vessel  does  indeed  impact  speed, 
maneuverability,  weight,  and  equipment  spacing  considerations.  Given  the  simplicity  of 
the  modeling  effort  at  hand,  ship  synthesis  considerations  were  not  accounted  for  in  the 
OSN  model,  since  its  primary  focus  was  on  agent  behavior  and  factor  significance. 

2.  MANA  Baseline  Model  Analysis 

Repeating  a  similar  analysis  process  to  the  data  generated  by  the  MANA  baseline 
model  helps  in  establishing  interchangeability.  Just  like  in  the  OSN  model  analysis, 
determining  overall  model  effectiveness  is  the  initial  focus. 


A  &  Response  MOE 


.a  Summary  of  Fit 


RSquare 

0.990288 

RSquare  Adj 

0.989323 

Root  Mean  Square  Error 

2.688327 

Mean  of  Response 

59.702 

Observations  (or  Sum  Wgts) 

200 

Analysis  of  Variance 

F  Ratio 

1025.359 

Prol)  >  F 

<.0001* 


Sinn  of 


Source 

DF 

Squares 

Mean  Square 

Model 

18 

133386.63 

7410.37 

Error 

181 

1308.10 

7.23 

C.  Total 

199 

134694.74 

Figure  23.  MANA  Baseline  Model  Regression 
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Since  the  data  are  averaged  much  in  the  same  way  as  the  OSN  data,  a  very  large 
portion  of  the  variance  in  the  data  is  accounted  for  by  the  regression  model,  as  shown  by 
the  R-Squared  of  99%.  This  results  in  the  similar  statement  that  the  regression  is  a  good 
predictor  of  model  behavior  in  regards  to  averaged  results.  Looking  at  the  Actual  by 
Predicted  Plot  qualitatively  begins  to  produce  observations  similar  to  the  OSN  data. 


Figure  24.  MANA  Baseline  Model  Actual  by  Predicted  Plot 


Formation  of  groupings  become  apparent,  just  as  in  the  OSN  data,  so  again  this 
points  to  key  factors  that  have  significant  impacts  on  simulation  performance. 


A  Sorted  Parameter  Estimates 

Term 

Estimate 

Std  Error 

t  Ratio 

Prol»|t| 

GUN[0] 

-23.82966 

0.193253 

-123.3 

<.0001* 

SSMfl] 

-7.879604 

0.191331 

-41.18 

L 

<.0001* 

GUN[0]*SSM[1] 

-3.111291 

0.192275 

-16.18 

:  i  T 

<.0001* 

SFACfl  ] 

3.1092659 

0.193651 

16.06 

]:  ; 

<.0001* 

GUN[0]*HELO|0] 

-2.185405 

0.19593 

-11.15 

<.0001* 

GUN[0)*SFAC[1] 

-2.021213 

0.192331 

-10.51 

] 

<.0001* 

Max  Speed 

0.3752795 

0.037235 

10.08 

<.0001* 

GUN[0f (Max  Speed-31) 

-0.339607 

0.036991 

-9.18 

<.0001* 

HELO[0]*SFAC[1] 

-1.660048 

0.193022 

-8.60 

<.0001* 

HELO[0] 

-1.595087 

0.192833 

-8.27 

<.0001* 

MastH 

0.460781 1 

0.057739 

7.98 

1: 

<.0001* 

HELO[0]*(Max  Speed-31) 

0.1473936 

0.037513 

3.93 

0.0001* 

HELO[0]*SSMfl] 

0.6670687 

0.19694 

3.39 

0.0009* 

(Max  Speed-31  )*SSM[1] 

0.1109279 

0.037152 

2.99 

0.0032* 

(MastH-17.5)*SSMfl] 

-0.157897 

0.059798 

-2.64 

0.0090* 

(MastH- 1 7.5)*(MastH-1 7.5) 

-0.055664 

0.021861 

-2.55 

0.0117* 

SFACfl  ]*SSM[1] 

0.490686 

0.193845 

2.53 

0.0122* 

GUN[0]*(MastH-1 7.5) 

0.1330198 

0.059346 

2.24 

0.0262* 

Figure  25.  MANA  Baseline  Model  Sorted  Parameter  Estimates 
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The  naval  gun  again,  appears  to  be  the  major  contributing  factor  in  regards  to  the 
response,  followed  by  the  SSM  factor.  However,  initial  observations  indicate  that  the 
helicopter  factor,  while  still  significant,  no  longer  carries  as  much  importance 
individually,  but  relies  more  so  on  its  interaction  with  other  factors.  This  maybe  the  result 
of  difference  in  computation  calculations  inherent  to  MANA  or  the  OSN  C++  coding,  or 
modeling  differences,  but  its  impact  is  addressed  later. 


Figure  26.  MANA  Baseline  Model  Prediction  Profiler  Plot 


a  ■*  Partition  for  MOE 


Figure  27.  MANA  Baseline  Model  Partition  Tree  Plot  (1st  Split) 
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a  ■»  Partition  for  MOE 
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Figure  28.  MANA  Baseline  Model  Partition  Tree  Plot  (2nd  Split) 


To  help  affirm  the  significance  of  the  naval  gun  and  SSM  factors,  like  in  the  OSN 
model,  prediction  profiler  and  partition  tree  plots  have  been  generated.  As  shown  in 
Figure  29,  the  naval  gun  factor  accounts  for  82.9%  of  all  variation  in  the  model,  slightly 
more,  but  similar  to  the  naval  gun  in  the  OSN  model.  The  SSM  factor  also  contributes  to 
a  fair  portion  of  the  variation  with  an  8.28%  increase  in  R-Squared,  similar,  but  again 
slightly  more  than  the  OSN  model. 
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Figure  29.  MANA  Baseline  Model  Actual  by  Predicted  Plot  (Gun  Absent  Selection) 

Addressing  the  groups  highlighted  in  Figure  26,  specific  selection  of  the  rows  of 
data  where  the  naval  gun  was  missing  from  the  simulation  generates  a  plot  similar  to 
Figure  20,  as  shown  by  Figure  31.  This  behavior  reflects  the  OSN  model,  and  the  same 
statement  can  be  made  that  regardless  of  the  other  factors,  exclusion  of  the  naval  gun 
from  the  simulation  model  results  with  an  MOE  in  the  lower  half  of  the  spectrum. 


Figure  30.  MANA  Baseline  Model  Actual  by  Predicted  Plot  (Gun  Absent  &  SSM  1 

Equipped) 
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Reflecting  similar  outcomes  to  the  OSN  model  in  Figure  22,  Figure  32  shows  that 
OP  Vs  not  equipped  with  a  naval  gun  and  employing  the  less  capable  missile  system 
performs  poorly  in  regards  to  the  ASUW  scenario  presented.  This  and  the  previous 
demonstrations  of  similarity  between  the  models  is  a  good  indication  that 
interchangeability  may  in  fact  be  possible,  but  there  are  differences  present,  as  shown  by 
additional  splits  of  the  partition  tree  for  the  MANA  baseline  data. 


Figure  3 1 .  MANA  Baseline  Model  Partition  Tree  (Splits  1-4) 
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Unlike  Figure  23,  which  focused  on  the  portion  of  data  missing  the  naval  gun 
factor,  the  MANA  baseline  model  partition  tree  begins  to  address  branches  for  both  factor 
levels.  When  the  naval  gun  is  absent  or  present,  the  missile  system  on  the  OPV  is  the  next 
most  important  factor,  but  following  this,  what  type  of  enemy,  either  SFAC  type  one  or 
type  two  is  more  important.  This  diverges  from  the  helicopter  factor  in  the  OSN  model, 
yet  comparing  the  factor  importance  between  Figure  17  and  Figure  27,  there  is  still  a 
close  similarity  between  the  two  models.  However,  this  comparison  is  insufficient  to 
ascertain  interchangeability.  To  accomplish  this  goal,  a  combination  of  both  data  sets  is 
required,  as  well  as  supplying  an  additional  factor  identifying  what  rows  of  data  originate 
from  which  sets  of  data.  This  factor  is  nothing  more  than  an  indicator  variable,  but  it 
holds  the  key  in  establishing  interchangeability. 

3.  Combined  Model  Data  Analysis 

Just  as  for  the  OSN  and  MANA  baseline  models,  the  combined  data  of  the  two  is 
be  used  to  generate  a  regression  model,  with  sorted  parameter  estimates,  partition  trees, 
and  prediction  profiles  to  help  identify  factor  significance. 
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Figure  32.  Combined  Data  Regression  (MANA  Data  Selected) 


For  ease  of  identification,  the  data  produced  by  the  MANA  baseline  model  have 
been  selected  in  Figure  34  and  Figure  35,  and  from  the  statistical  output  it  seems  that  the 
regression  model  is  accounting  for  a  very  large  portion  of  the  variance  based  upon  the 
high  R-squared  value.  This  behavior  is  present  in  both  regressions  of  the  individual  data 
sets,  and  since  both  sets  of  data  feed  responses  that  are  averaged,  it  is  reasonable  to 
assume  that  the  resulting  combination  of  the  data  would  produce  a  regression  model 
accounting  for  most  of  the  variation  as  well. 
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Figure  33.  Combined  Data  Actual  by  Predicted  Plot 


A  qualitative  look  at  the  actual  by  predicted  plot  in  Figure  35  seems  to  indicate  a 
shift  of  the  MANA  baseline  data  toward  better  results  that  the  OSN  data.  To  investigate 
this  occurrence,  identifying  the  impact  of  the  indicator  variable  previously  mentioned 
needs  to  take  place. 
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(Max  Speed-31  )*l  n  d  i  c  ato  r[lta  Man] 
SFAC[1]*SSM[1] 

(MastH-1 7. 5)*  (MastH-1 7.5) 
(MastH-1 7.5)*lndicator[ltalian] 


Estimate 

Std  Error 

t  Ratio 

-23.59289 

0.064146 

-367.8 

-7.714312 

0.064142 

-120.3 

3.2859993 

0.064143 

51.23 

-2.849789 

0.064161 

-44.42 

-3.63687 

0.109896 

-33.09 

-1.999243 

0.064145 

-31.17 

-3.422189 

0.109856 

-31.15 

-0.345414 

0.011262 

-30.67 

-1.694211 

0.064163 

-26.40 

-1.551592 

0.064174 

-24.18 

0.4513485 

0.020873 

21.62 

-0.214193 

0.011262 

-19.02 

-1.885728 

0.109856 

-17.17 

1.0823745 

0.064156 

16.87 

0.3584559 

0.03253 

11.02 

-0.021844 

0.002239 

-9.76 

-0.093589 

0.011262 

-8.31 

0.1208131 

0.019122 

6.32 

0.0632971 

0.011265 

5.62 

-0.086871 

0.019135 

-4.54 

0.093904 

0.020873 

4.50 

0.2612681 

0.0642 

4.07 

-0.021992 

0.007131 

-3.08 

-0.093817 

0.03253 

-2.88 

Piol»|t| 

<  0001* 
<.0001* 
<.0001* 
<.0001* 
<.0001* 
<.0001* 
<.0001* 
<.0001* 
<.0001* 
<.0001* 
<.0001* 
<.0001* 
<.0001* 
<.0001* 
<.0001* 
<.0001* 
<.0001* 
<.0001* 
<.0001* 
<.0001* 
<.0001* 
<.0001* 
0.0021* 
0.0040* 


Figure  34.  Combined  Data  Sorted  Parameter  Estimates 
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Just  like  with  the  individual  analysis  of  the  baseline  datasets,  the  sorted  parameter 
estimates  shown  in  Figure  36  indicate  that  again  the  naval  gun  and  SSM  factors  play  as 
the  most  significant  factors  in  the  combined  model.  However,  the  indicator  variable, 
annotated  Indicator[Italian]  places  fifth  in  importance  to  other  factors,  which  means  any 
difference  between  the  models  is  dominated  by  four  factor  effects,  including  the 
helicopter  due  to  its  interactions. 


Figure  35.  Leverage  Plot  for  Indicator  Variable 

Figure  37  shows  there  exists  a  statistically  significant  difference  between  both 
sets  of  data,  both  qualitatively  and  quantitatively  since  the  least  square  means  of  both 
data  sets  differs  by  7.27.  This  improved  performance  by  the  MANA  baseline  model  can 
be  the  result  of  a  whole  host  of  different  factors,  be  they  programming  differences, 
behaviors  in  hit  probability  calculations,  or  agent  targeting  criteria  inherent  to  MANA  or 
the  C++  code.  Whatever  the  reason  behind  the  difference  in  mean  responses  between  the 
two  data  sets  is  actually  irrelevant.  What  is  important  is  model  behavior.  As  previously 
shown,  both  models  behave  very  closely  in  regards  to  key  factor  importance,  but  the  true 
test  in  determining  if  the  models  are  behaving  similarly  is  by  analyzing  the  interactions  of 
factors  with  the  indicator  variable.  If  no  interactions  exist,  then  the  difference  between 
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the  two  models  can  easily  be  removed  by  the  inclusion  of  a  correction  factor  to  help  shift 
one  regression  onto  the  other,  ie;  the  indicator  variable.  If  interactions  with  the  indicator 
variable  do  exist  however,  then  the  problem  becomes  more  complicated  since  the  slopes 
of  the  regression  lines  are  different,  and  it  is  possible  to  state  that  the  change  in  model 
response  is  a  function  of  which  data  set  being  viewed.  This  outcome  is  undesirable,  so 
scrutinizing  the  sorted  parameter  estimates  for  indicator  interactions  is  necessary. 


HELO[0]*(Max  Speed-31) 

-0.214193 

0.011262 

-19.02 

<.0001* 

HELOIOrindicatorlltalianl 

-1.885728 

0.109856 

-17.17 

<  0001* 

HELO[0]*SSM[1] 

1.0823745 

0.0641 56 

16.87 

<  0001* 

MastH 

0.3584559 

0.03253 

11.02 

<.0001* 

(Max  Speed-31)*(Max  Speed-31) 

-0.021844 

0.002239 

-9.76 

<  0001* 

(Max  Speed-31  )*SFAC[1] 

-0.093589 

0.011262 

-8.31 

<  0001* 

HELO[0]*(MastH-1 7.5) 

0.1208131 

0  019122 

6.32 

<  0001* 

(Max  Speed-31  )*SSM[1] 

0.0632971 

0.011265 

5.62 

<  0001* 

(MastH-1 7.5)*SSM[1] 

-0.086871 

0.019135 

-4.54 

<.0001* 

(Max  Speed-31  )*lndicator[ltalianl 

0.093904 

0.020873 

4.50 

<  0001* 

SFAC[1  ]*SSM[1] 

0.2612681 

0.0642 

4.07 

<.0001* 

(MastH-1 7.5)*(MastH-1 7.5) 

-0.021992 

0.007131 

-3.08 

0.0021* 

(MastH-1 7.5V*lndicatorfltalianl 

-0.09381 7 

0.03253 

-2.88 

0.0040* 

Figure  36.  Excerpt  From  Sorted  Parameter  Estimates  of  Combined  Data 


As  shown  in  Figure  38,  there  exist  three  statistically  significant  interaction  terms 
with  the  indicator  variable.  With  this  in  mind,  analysis  of  their  impact  on  the  combined 
model  can  be  performed  to  better  understand  their  influence  on  model  behavior. 


R-Squard 

Delta 

Full  Model 

0.987712 

Indicator  Interaction  Factors  Removed 

Helicopter  &  Indicator 

0.985984 

0.001728 

Mast  Height  &  Indicator 

0.987664 

4.8E-05 

Speed  &  Indicator 

0.987594 

0.000118 

Helicopter  &  Indicator, 
Mast  Height  &  Indicator 

0.985935 

0.001777 

Helicopter  &  Indicator, 
Speed  &  Indicator 

0.985866 

0.001846 

Mast  Height  &  Indicator, 
Speed  &  Indicator 

0.987545 

0.000167 

ALL  Indicator  Interactions 

0.985847 

0.001865 

Table  4.  Impact  on  R-Squared  With  Indicator  Interactions  Removal 
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Since  Figure  38  shows  that  there  are  statistically  significant  interactions  terms 
with  the  indicator  variable,  systematically  removing  these  interactions  from  the  combined 
dataset  model  can  be  carried  out  to  understand  their  impact  on  accounting  for  model 
variation.  Table  4  shows  the  indicator  interaction  removal  process,  from  each  individual 
interaction  removed  to  all  three  interactions,  and  every  combination  in  between.  The 
Delta  column  represents  the  difference  in  R-Squared  from  the  full  model,  to  a  model  with 
the  removed  interaction  terms  prescribed.  The  end  result  from  analyzing  Table  4  is  that 
even  if  all  indicator  interaction  terms  are  removed  from  the  combined  model,  they 
account  for  less  than  0.2%  of  all  variation  explained  by  the  regression.  This  ultimately 
means  that  while  statistically  significant,  the  indicator  interaction  terms  are  negligible  in 
relation  to  model  behavior  and  performance,  and  with  the  inclusion  of  the  indicator 
variable  for  correction  purposes,  the  two  baseline  models  can  be  deemed  as 
interchangeable.  With  this  condition  established,  factor  and  interaction  significance  can 
be  determined  to  establish  the  basis  for  the  advanced  model  comparisons. 

4.  Baseline  Factor  Significance 

With  the  establishment  that  both  the  MANA  and  OSN  baseline  models  can  be 
used  interchangeably,  the  next  focus  of  the  analysis  is  the  breakdown  of  factor 
significance  on  prosecuting  SFAC  agents.  This  means  a  shift  from  the  MOE  of  successful 
goal  line  defense,  to  the  MOP  of  OPV  factors  that  impact  SFAC  termination. 
Unfortunately,  the  OSN  data  does  not  include  the  number  of  SFACs  tenninated  for  a 
single  simulation  run;  however,  the  MANA  baseline  data  does,  and  the  average  number 
of  SFAC  killed  becomes  the  new  measure  of  performance  for  the  simulated  OPVs.  The 
analysis  focuses  on  each  of  the  specific  factors  utilized  in  the  model  and  any  particular 
interactions  that  significantly  account  for  variation  in  the  model. 


63 


a. 


Naval  Gun 


A  ▼  OPV  Gun  Present 


Figure  37.  Baseline  Naval  Gun  Leverage  Plot 


From  Figure  39,  the  naval  gun  is  shown  to  still  be  the  most  significant 
factor  in  determining  OPV  performance  in  regards  to  SFAC  kills.  The  tight  grouping  to 
the  left  represents  OPV  performance  the  naval  gun’s  absence,  while  the  right  grouping 
represents  its  availability. 

b.  Surface  to  Surface  Missile  Type 


0.8  1.0  1.2  1.4  1.6  1.8  2.0 

OPV  Missile  Type 
Leverage,  P<.0001 


Figure  38.  Baseline  SSM  Leverage  Plot 
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Like  the  naval  gun,  SSM  configuration  on  the  OPV  plays  a  huge  part  on 
perfonnance  in  the  ASUW  scenario.  This  impact  is  largely  due  to  the  higher  hit  accuracy 
of  the  missile  systems  relative  to  the  naval  gun.  Unfortunately  SSM  performance  can  be 
limited  by  the  exclusion  of  the  helicopter  from  a  simulation  run,  unlike  the  naval  gun. 
Additionally,  after  the  expenditure  of  all  missiles  available  on  the  OP  Vs  have  taken 
place,  if  SFACs  still  remain  on  the  battlefield  then  the  naval  gun  is  the  only  possible 
alternative  to  achieve  mission  success.  This  is  why  the  gun  is  plays  a  larger  role  than  the 
missile  system,  even  though  the  majority  of  SFAC  kills  have  occurred  due  to  missile 
strikes. 


c.  SFAC  Type 


A  r]  SFAC  Type 


Figure  39.  SFAC  Type  Leverage  Plot 


While  providing  not  as  quite  a  drastic  impact  on  the  MOP  of  number  of 
SFAC  kills,  the  SFAC  type  factor  does  play  a  big  part  in  simulation  results.  The  type  of 
SFAC  the  OP  Vs  are  dealing  with  greatly  increases  or  decreases  the  radar  perfonnance 
since  it  is  inherently  harder  to  detect  smaller  objects  at  sea.  Additionally,  larger  SFACs 
are  capable  of  mounting  more  powerful  engines,  and  can  potentially  outrun  defending 
OPVs.  The  end  result  is  a  decrease  in  mean  response  of  6.57.  This  passes  the  common 
sense  test  that  OPV  effectiveness  should  degrade  when  combating  a  swifter  foe. 
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d.  OPV Radar  Height 


A  ▼  OPV  Radar  Heights 


Figure  40.  OPV  Radar  Height  Leverage  Plot 


As  shown  by  Figure  42,  the  leverage  of  OPV  radar  height  is  low  relative 
to  the  gun  or  type  of  SSM.  While  having  an  effective  surface  search  radar  is  important  to 
detect  the  small  boat  threat,  for  this  particular  simulation,  effects  such  as  poor  sea-state  or 
curvature  of  the  earth  were  not  modeled,  so  radar  height  relatively  important.  This  may 
not  be  the  case  in  a  simulation  that  accounts  for  such  noise  variables. 

e.  Helicopter  Presence 


A  -  Helo  Present 


Figure  4 1 .  Helicopter  Presence  Leverage  Plot 
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Similar  in  behavior  to  the  OPV  radar  height  factor,  the  helicopter  provided 
additional  tactical  range  to  the  OPVs  in  their  detection  and  prosecution  of  approaching 
SFACs.  Since  issues  such  as  weather  or  line  of  sight  issues  were  not  considered,  the 
importance  of  the  helicopter  may  be  being  undennined.  However,  the  helicopter  did 
increase  OPV  performance  in  killing  SFACs,  which  was  expected. 

f.  OPV  Max  Speed 

a  -  OPVMaxSpeed 


a  Leverage  Plot 


Figure  42.  OPV  Max  Speed  Leverage  Plot 


Once  SFACs  are  detected,  the  maximum  speed  of  OPVs  has  some,  but  not 
a  lot,  of  impact  on  SFAC  kills,  as  shown  in  Figure  44.  This  outcome  has  more  to  do  with 
the  capabilities  of  the  equipped  missile  systems,  detection  ranges,  and  behaviors  of  the 
OPV  agents  in  the  simulation.  The  reason  is  that  in  many  situations,  a  slower  but  better 
equipped  OPV  may  be  at  least  as  effective  as  a  faster,  but  less  equipped  one.  In  general,  a 
faster  OPV  means  that  it  can  get  in  range  quicker  to  kill  approaching  SFACs,  but  it  first 
has  to  detect  them.  It  then  may  have  weapons  capable  of  striking  targets  immediately, 
regardless  of  maximum  speed. 
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D.  ADVANCED  MODEL  ANALYSIS 


1.  Avoidance  SFACs  vs.  Baseline  OPVs 

As  stated  in  the  methodology  section,  in  the  avoidance  situation  the  SFACs  are 
attempting  to  circumvent  their  OPV  opponents.  Since  the  number  of  SFACs  involved  has 
been  doubled,  the  MOE  of  goal  line  defense  has  been  removed,  while  focusing  on  OPV 
MOPs  has  become  more  important. 


zi  Sorted  Parameter  Estimates 


Term 

OPV  Gun  Present[0) 

OPV  Missile  Type[1] 

OPV  Gun  Present[0]*SFAC  Type[1] 

SFAC  Type[1] 

OPV  Gun  Present[0]*(OPVMaxSpeed-30.971 4) 
OPVMaxSpeed 

SFAC  Typed  ]*(OPVMaxSpeed-30.971 4) 

OPV  Gun  Present[0]*OPV  Missile  Type[1] 


Estim.ite 

Std  Error 

t  Ratio 

-2.279607 

0.023769 

-95.90 

-0.517195 

0.023862 

-21.67 

-0.344224 

0.023792 

-14.47 

0.3113671 

0.023792 

13.09 

-0.053382 

0.004583 

-11.65 

0.0494545 

0.004573 

10.82 

0.0192995 

0.00462 

4.18 

-0.093197 

0.023779 

-3.92 

Proh>|t| 

<.0001* 
<  0001* 
<.0001* 
<.0001* 
<.0001* 
<.0001* 
<.0001* 
0.0001* 


Figure  43.  Avoidance  vs.  Baseline  OPV  Sorted  Parameter  Estimates 


Like  in  previous  analyses,  missing  the  naval  gun  and  having  the  Marte  missile 
system  equipped  has  the  largest  negative  impact  on  performance.  Additionally,  the  type 
of  SFAC  the  OPVs  are  combating  seems  to  have  a  larger  impact  since  model 
performance  seems  to  increase  when  OPVs  are  facing  the  slower  type  1  SFAC.  What 
seems  to  be  more  interesting  is  that  the  helicopter  factor  has  been  removed  from  model 
significance.  This  means  that  factors  associated  specifically  to  the  OPV  and  the  SFACs 
are  what  are  most  important  for  the  MOP,  and  not  the  detection  capability  provided  by 
the  helicopter. 

2.  Aggressive  SFACs  vs.  Baseline  OPVs 

If  the  SFAC  avoidance  behavior  is  reversed  the  SFACs  become  aggressive  to  the 
OPVs.  The  aggressive  scenario  tries  to  show  OPV  defensive  performance  in  a  hostile 
SFAC  environment. 
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a  Sorted  Parameter  Estimates 

Term 

Estimate 

Std  El  ioi 

t  Ratio 

Piob>|t| 

OPV  Gun  Present[0] 

-1.271576 

0.017744 

-71.66 

<0001* 

OPV  Missile  Type[1] 

-0.44646 

0.017564 

-25.42 

L 

<0001* 

Helo  Present[0] 

-0.240027 

0.017509 

-13.71 

<.0001* 

SFAC  Type[1]*(OPVMaxSpeed-30.971 4) 

-0.020517 

0.003373 

-6.08 

<.0001* 

Helo  PresenWSFAC  Type[1] 

-0.106842 

0.017611 

-6.07 

<0001* 

SFAC  Type [1] 

-0.097253 

0.017422 

-5.58 

<0001* 

OPV  Gun  Present[0]*OPV  Missile  Type[1] 

-0.091033 

0.017621 

-5.17 

1 

<.0001* 

Helo  Present[0]*OPV  Gun  Present[0] 

0.0834585 

0.017467 

4.78 

<.0001* 

OPV  Radar  Heights{1 3-22}*SFAC  Type[1  ] 

-0.084997 

0.024461 

-3.47 

0.0006* 

OPV  Radar  Heights{1 3-22} 

-0.076548 

0.024339 

-3.15 

0.0019* 

OPVMaxSpeed 

0.0098877 

0.003348 

2.95 

0.0036* 

OPV  Radar  Heights{1 6&1 9-1 3&22> 

-0.018079 

0.017929 

-1.01 

0.3146 

Figure  44.  Aggressive  vs.  Baseline  Sorted  Parameter  Estimates 


Figure  46  displays  how  the  lack  of  a  naval  gun  and  use  of  the  less  capable  missile 
system  yields  lower  and  lower  measures  of  performance.  Unlike  in  the  Avoidance 
scenario  however,  the  helicopter  factor  returns  to  be  the  third  most  significant  factor 
followed  by  the  interaction  term  of  SFAC  type  and  OPV  maximum  speed.  The  individual 
OPV  maximum  speed  factor  dropped  significantly  in  its  contribution  to  OPV 
performance.  The  reason  behind  this  observation  is  that  SFACs  are  now  closing  and 
engaging  detected  OP  Vs,  which  means  OP  Vs  are  less  likely  to  need  to  pursue  their  small 
boat  targets. 


3.  Combination  Avoidance  &  Aggressive  SFACs  vs.  Baseline  OPVs 

The  combination  scenario  introduces  both  the  avoidance  and  aggressive  SFAC 
agents  to  the  simulation,  and  attempts  to  see  how  these  varied  behaviors  impact  OPV 
factor  performance.  This  approach  of  a  mixed  SFAC  threat  has  substantial  credibility 
since  strategies  like  these  were  employed  by  the  Tamil  Sea  Tigers  to  break  Sri  Lankan 
naval  blockades  near  the  end  of  their  civil  war.  Suicide  attack  craft  would  engage 
blockading  forces  in  an  attempt  to  get  resupply  ships  through.  Additionally,  this  may  be  a 
strategy  employed  by  an  asymmetric  adversary  to  attack  guarded,  high  value  targets,  such 
as  naval  aircraft  carriers  or  oil  platforms. 


69 


a. 


Avoidance  SFAC Prosecution 


A  Sorted  Parameter  Estimates 


Term 

OPV  Missile  Type[1] 

OPV  Gun  Present[0] 

OPV  Radar  Heights{22&1 9-1 6&1 3} 

(OPV  Radar  Heights{22&1 9-1 6&1 3}+0.00503)*OPV  Missile  Type[1  ] 
SFAC  Type[1] 

OPV  Missile  Type[1  ]*SFAC  Type[1  ] 

(OPV  Radar  Heights{22&1 9-1 6&1 3}+0.00503)*SFAC  Type[1] 

(OPV  Radar  Heights{22-1 9J-0.01 508)*OPV  Missile  Type[1] 

OPV  Radar  Heights{22-1 9} 

OPVMaxSpeed 

OPV  Gun  Present[0]*(OPVMaxSpeed-30.971 4) 

SFAC  Type[1  ]*(OPVMaxSpeed-30.971 4) 

(OPV  Radar  Heights{22&1 9-1 6&1 3}+0.00503)*OPV  Gun  PresentfO] 
OPV  Gun  Present[Q]*OPV  Missile  Type[1] 


Estimate 

Std  Error 

-0.491378 

0.016229 

-0.448231 

0.01629 

-0.195481 

0.016256 

-0.191758 

0.016478 

0.1880949 

0.01627 

0.1201119 

0.016317 

0.1112287 

0.016335 

-0.118707 

0.023439 

-0.118177 

0.023467 

0.0130586 

0.003165 

-0.012556 

0.003147 

-0.012294 

0.0032 

-0.039126 

0.016382 

-0.012809 

0.016289 

t  Ratio 

-30.28 

-27.52 

-12.02 

-11.64 

11.56 

7.36 

6.81 

-5.06 

-5.04 

4.13 

-3.99 

-3.84 

-2.39 

-0.79 


Prol»|t| 

<0001* 

<.0001* 

<.0001* 

<.0001* 

<.0001* 

<.0001* 

<.0001* 

<.0001* 

<.0001* 

<.0001* 

<.0001* 

0.0002* 

0.0179* 

0.4327 


Figure  45.  Combination  vs.  Baseline:  Avoidance  SFAC  Prosecution  Sorted 

Parameter  Estimates 


In  the  combination  scenario,  factor  importance  takes  a  turn  in  significance 
from  the  naval  gun  to  the  equipped  missile  type  when  focusing  on  avoidance  SFACs. 
Additionally,  the  OPV  radar  height,  and  its  interaction  with  the  missile  type  factor  move 
up  in  importance  for  model  performance.  Reasons  behind  this  change  are  due  to  the 
simulation  environment,  where  OPVs  initial  engagement  on  avoidance  SFACs  is  largely 
dependent  on  missile  attacks  and  detection  distance.  As  the  simulation  progresses,  OPVs 
become  hindered  and  overwhelmed  by  the  attacking  aggressive  SFACs,  and  they  no 
longer  have  the  resources  or  ability  to  prosecute  the  avoiding  SFACs  to  keep  them  from 
reaching  the  goal  line. 


b.  Aggressive  SFAC  Prosecution 


a  Sorted  Parameter  Estimates 

Term 

Estimate 

Std  Ei  i  or 

t  Ratio 

Piol»|t| 

OPV  Gun  Present(C)] 

-0.510819 

0.011059 

-46.19 

<.0001* 

OPV  Radar  Heights{1 6&1 9&1 3-22} 

-0.108935 

0.01286 

-8.47 

: :  V 

<  0001* 

(OPV  Radar  Heights{1 6&1 9&1 3-22}-0.48744)*OPV  Missile  Type[1] 

-0.08246 

0.012529 

-6.58 

<.0001* 

SFAC  Type[1] 

-0.071093 

0.01095 

-6.49 

<.0001* 

OPV  Missile  Typefl] 

-0.070638 

0.01093 

-6.46 

<.0001* 

Helo  Present[0] 

-0.055297 

0.011061 

-5.00 

i  I 

<.0001* 

OPV  Radar  Heights{1 9-1 3} 

0.0740784 

0.01577 

4.70 

li  i 

<.0001* 

OPV  Missile  Type[1]*SFAC  Type[1] 

-0.042504 

0.01105 

-3.85 

0.0002* 

Helo  Present[0]*SFAC  Typefl] 

-0.041385 

0.011013 

-3.76 

0.0002* 

(OPV  Radar  Heights{1 6&1 9&1 3-22}-0.48744)*OPV  Gun  Present[0] 

-0.045356 

0.012941 

-3.50 

0.0006* 

(OPV  Radar  Heights{1 6-1 9&1 3}+0.251 26)*OPV  Gun  Present[0] 

-0.037262 

0.013752 

-2.71 

0.0074* 

(OPV  Radar  Heights{1 9-1 3}+0.01 508)*Helo  Presents 

0.0337595 

0.015565 

2.17 

0.0314* 

OPV  Radar  Heights{1 6-1 9&1 3} 

-0.024462 

0.013775 

-1.78 

0.0774 

Figure  46.  Combination  vs.  Baseline:  Aggressive  SFAC  Prosecution  Sorted 

Parameter  Estimates 
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In  contrast  to  the  factor  importance  shown  in  Figure  47,  Figure  48  shows  a 
return  to  the  importance  of  the  naval  gun  for  the  aggressive  SFAC  prosecution  portion  of 
the  scenario.  OPV  radar  height  and  its  interaction  with  SSM  type  still  play  a  large  role  in 
determining  performance.  As  with  the  avoidance  SFACs,  the  first  engagement  of  the 
OP  Vs  against  the  aggressive  threat  is  via  missile  strikes,  but  as  the  problem  progresses  to 
a  more  close  quarters  fight,  OP  Vs  are  forced  to  rely  almost  entirely  upon  their  naval  gun 
if  equipped  to  fend  off  the  suicide  aggressive  SFACs.  If  the  gun  is  not  equipped,  OPVs 
are  essentially  defenseless  once  all  missiles  are  expended,  and  are  destroyed. 


c.  Avoidance  &  Aggressive  SFAC  Prosecution 


A  Sorted  Parameter  Estimates 

Tenn 

Estimate 

Std  El  ioi 

t  Ratio 

Pi  ol»|t| 

OPV  Gun  Present[0] 

-0.953348 

0.015974 

-59.68 

<0001* 

OPV  Missile  Type[1) 

-0.547443 

0.015841 

-34.56 

<  0001* 

SFAC  Type[1  ] 

0.1162076 

0.01573 

7.39 

]:  .  . 

<  0001* 

(OPV  Radar  Heights{1 9&1 6&22-1 3)-0.48744)*OPV  Missile  Type[1  ] 

-0.108507 

0.018897 

-5.74 

i  \ 

<  0001* 

(OPV  Radar  Heights{1 9&1 6-22J-0.231 1 6)*OPV  Missile  Type|1] 

0.1029935 

0.01945 

5.30 

<  0001* 

OPV  Missile  TypeflfSFAC  Type[1] 

0.0831372 

0.015821 

5.25 

<.0001* 

SFAC  Type[1f  (OPVMaxSpeed-30.971 4) 

-0.016195 

0.003135 

-5.17 

<  0001* 

Helo  Present[0] 

-0.078944 

0.015813 

-4.99 

<  0001* 

OPV  Gun  Present|0]*(OPVMaxSpeed-30.971 4) 

-0.014152 

0.003064 

-4.62 

<  0001* 

OPV  Radar  Heights{1 9&1 6-22} 

0.0874424 

0.019373 

4.51 

<  0001* 

(OPV  Radar  HeightS{1 9&1  6&22-1 3)-0.48744)*SFAC  Type[1  ] 

0.081443 

0.018477 

4.41 

<  0001* 

OPV  Radar  Heights{1 9&1 6&22-1 3} 

-0.080281 

0.018328 

-4.38 

<  0001* 

OPVMaxSpeed 

0.0130101 

0.003095 

4.20 

<  0001* 

Helo  Present[0]*S FAC  Typed) 

-0.059155 

0.015854 

-3.73 

0.0003* 

(OPV  Radar  Heights{1 9-1 6}+0.00503)’SFAC  Type[1  ] 

0.0855746 

0.023077 

3.71 

0.0003* 

(OPV  Radar  Heights{1 9&1 6-22J-0.231 1 6)*SFAC  Type[1) 

-0.072091 

0.019454 

-3.71 

0.0003* 

(OPV  Radar  Heightsfl  9-1 6}+0.00503)*OPV  Missile  Typed) 

-0.080712 

0.023138 

-3.49 

0.0006* 

OPV  Radar  Heights{1 9-1 6} 

-0.07792 

0.023062 

-3.38 

0.0009* 

OPV  Gun  Pres ent[0fS FAC  Typed) 

-0.049668 

0.016034 

-3.10 

0.0023* 

(OPV  Radar  Heights{1 9-1 6}+0.00503)*Helo  Presente] 

0.0655065 

0.023158 

2.83 

0.0052* 

(OPV  Radar  Heights{1 9&1 6-22}-0.231 1 6)*HelO  Presente] 

0.0510729 

0.019161 

2.67 

0.0084* 

OPV  Gun  Presentei*OPV  Missile  Typed) 

-0.034508 

0.01631 

-2.12 

0.0358* 

Helo  Presenter(OPVMaxSpeed-30.971 4) 

0.0003932 

0.003086 

0.13 

0.8988 

Figure  47.  Combination  vs.  Baseline  Sorted  Parameter  Estimates 


When  the  previous  two  models  resulting  in  Figure  47  and  Figure  48  are 
combined,  the  subsequent  factor  significance  is  displayed  in  Figure  49.  The  naval  gun 
still  remains  the  most  influential  factor  in  determining  OPV  performance  in  the  full 
combination  scenario,  but  equipped  SSM  type  is  slightly  more  significant  than  in 
previous  model  scenarios,  as  in  the  specific  avoidance  or  aggressive  models.  OPV  radar 
height  and  interactions  with  missile  type  are  very  important  in  this  environment  since 
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they  largely  dictate  how  the  scenario  proceeds  at  the  beginning  of  a  simulation  run,  which 
in  turns  helps  to  dictate  how  difficult  the  problem  becomes  for  OP  Vs  once  SFACs  begin 
to  engage  or  avoid  them.  This  particular  scenario  is  somewhat  more  interesting  than 
previous  ones,  because  it  begins  to  explore  a  more  realistic  threat  in  regards  to  the  small 
boat  swarm  attack. 


4.  Avoidance  SFACs  vs.  Enhanced  OPVs 

Given  the  challenging  environment  the  baseline  OPVs  were  placed  in  for  the 
previous  model  scenarios,  it  is  only  fair  to  explore  simulation  outcomes  with  a  more 
robust  and  capable  OPV.  These  enhanced  OPVs  have  several  changes  made  to  them  to 
help  them  better  combat  the  avoidance  SFAC  threat,  which  are  detailed  in  Chapter  III. 


a  Sorted  Parameter  Estimates 


Teim 

SFAC  Type[1] 

OPVMaxSpeed 
Helo  Present[0] 

SFAC  Type[1]*(OPVMaxSpeed-30.971 4) 

OPV  Missile  Type[1] 

Helo  Present[0]*SFAC  Type[1] 

(OPV  Radar  Heights{1 9&1 3-22J-0.241 21)*SFAC  Type[1  ] 
(OPVMaxSpeed-30.971 4)*(OPVMaxSpeed-30.971 4) 

OPV  Oun  Present[0] 

(OPV  Radar  Heights{1 9-1 3}+0.01 508)*SFAC  Type[1] 

Helo  Present[0]*(OPVMaxSpeed-30.971 4) 

(OPV  Radar  Heights{1 9&1 3&22-1 6}-0.50754)*(OPVMaxSpeed-30.971 4) 
(OPV  Radar  Heights{1 9-1 3J+0.01  508)*(OPVMaxSpeed-30.971 4) 

OPV  Radar  Heights{1 9&1 3-22} 

OPV  Radar  Heights{1 9&1 3&22-1 6} 

Helo  Present[0]’OPV  Missile  Type[1] 

OPV  Radar  Helghts{1 9-13} 


Estimate 

Std  El  nil 

t  Ratio 

0.7224214 

0.029458 

24.52 

0.1294463 

0.005619 

23.04 

0.2939655 

0.029363 

10.01 

0.0466665 

0.005688 

8.20 

-0.192788 

0.029732 

-6.48 

-0.190675 

0.029438 

-6.48 

-0.164899 

0.035272 

-4.68 

-0.004724 

0.00121 

-3.90 

-0.113292 

0.029568 

-3.83 

0.1512115 

0.042536 

3.55 

-0.018707 

0.005642 

-3.32 

0.0206967 

0.006861 

3.02 

0.0234651 

0.00786 

2.99 

-0.032904 

0.0355 

-0.93 

-0.015526 

0.03509 

-0.44 

-0.011524 

0.029456 

-0.39 

-0.015798 

0.041526 

-0.38 

Pi  ol»|t| 

<0001* 

<.0001* 

<.0001* 

<  0001* 

<.0001* 

<.0001* 

<  0001* 

0.0001* 

0.0002’ 

0.0005* 

0.0011* 

0.0029* 

0.0032* 

0.3552 

0.6587 

0.6961 

0.7041 


Figure  48.  Avoidance  vs.  Enhanced  Sorted  Parameter  Estimates 


With  the  modifications  to  the  naval  gun  being  in  effect,  its  significance  drops 
sharply  for  OPV  perfonnance,  and  now  the  problem  is  more  reliant  upon  the  SFAC  type 
and  OPV  maximum  speed.  This  result  has  more  credibility  when  referencing  documents 
concerning  the  Sri  Lankan  Civil  War  and  the  Iranian  Naval  Warfare  Doctrine 
(Haghshenass,  2006).  The  helicopter  factor  also  plays  a  large  role  in  OPV  performance, 
because  tactical  data  being  fed  to  the  OPVs  helps  to  dictate  their  motion  given  the  need  to 
pursue  the  avoiding  SFAC  threat.  This  result  diverges  from  the  previous  avoidance 


scenario  which  excluded  the  helicopter  factor. 
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5. 


Aggressive  SFACs  vs.  Enhanced  OPVs 


a  Sorted  Parameter  Estimates 


Teim 

OPV  Radar  Heights{1 3-1 6&1 9&22} 

(OPV  Radar  Heights{1 3-1 6&1 9&22}+0.48744)*SFAC  Type[1] 
SFAC  Type[1] 

(OPV  Radar  Heights{1 3-1 6&1 9&22}+0.48744)*Helo  Present[0] 
Helo  Present[0] 

Helo  Present[0]*SFACType[1] 


Estimate 

Std  El  ioi 

t  Ratio 

-0.0041 1 7 

0.000562 

-7.32 

-0.003752 

0.00056 

-6.70 

-0.002777 

0.000487 

-5.70 

-0.002787 

0.000562 

-4.96 

-0.002075 

0.000489 

-4.25 

-0.002019 

0.000489 

-4.13 

Pi  ol»|t| 

<0001* 

<0001* 

<0001* 

<.0001* 

<.0001* 

<.0001* 


Figure  49.  Aggressive  vs.  Enhanced  Sorted  Parameter  Estimates 


Quite  possibly  the  most  surprising  outcome  of  the  enhanced  OPV  scenarios  is  the 
aggressive  SFAC  threat.  As  seen  in  Figure  51,  the  only  contributing  factor  specific  to  the 
OPV  is  its  detection  capability,  represented  by  the  radar  height  and  helicopter  factor.  The 
helicopter  is  less  important  that  the  OPV  radar  height.  This  result  is  similar  to  the 
previous  aggressive  SFAC  scenario,  because  OPVs  are  not  required  to  pursue  SFAC 
threats  as  much  as  in  the  avoidance  scenarios,  since  SFACs  detect  and  attempt  to  collide 
with  OPVs.  Also,  since  the  Griffin  missile  system  is  included  on  all  OPVs  its 
effectiveness  is  driving  down  the  significance  of  the  surface  to  surface  missile  capability 
and  the  naval  gun.  While  the  assumptions  made  about  the  effectiveness  of  the  Griffin 
missile  system  are  still  under  evaluation,  a  comparable  weapon  system  proposed  in 
Cobian  (2002),  may  have  a  similar  impact  on  OPV  performance. 
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6. 


Combination  Avoidance  &  Aggressive  SFACs  vs.  Enhanced  OPVs 


a.  Avoidance  SF AC  Prosecution  (Enhanced  OPVs) 


A  Sorted  Parameter  Estimates 

Tenn 

Estimate 

Stcl  El  ioi 

t  Ratio 

Pi  ob>|t| 

SFACType[1] 

0.7723136 

0.019143 

40.34 

<.0001* 

OPV  Missile  Type[1] 

-0.440996 

0.01926 

-22.90 

1 

<.0001* 

OPVMaxSpeed 

0.0763678 

0.003754 

20.34 

<.0001* 

OPV  Missile  Type[1]*SFAC  Type[1] 

0.2378455 

0.019673 

12.09 

I Ti 

<  0001* 

(OPV  Radar  Heights{22-1 3&1 9&1 6}+0.48744)*OPV  Missile  Type[1] 

-0.257776 

0.022749 

-11.33 

□ 

<  0001* 

(OPV  Radar  Heights{1 9-1 6}+0.00503)*SFAC  Type[1] 

0.2185977 

0.028427 

7.69 

j  ! 

<.0001* 

SFAC  Type|1]*(OPVMaxSpeed-30.971 4) 

-0.028068 

0.003805 

-7.38 

<.0001* 

OPV  Radar  Heights{22-1 3&1 9&1 6} 

-0.156836 

0.022052 

-7.11 

<.0001* 

OPV  Gun  Present[0] 

-0.130516 

0.019467 

-6.70 

<.0001* 

(OPV  Radar  Heightsjl  9-1 6)+0.00503)*OPV  Missile  Type[1] 

-0.143313 

0.028478 

-5.03 

<.0001* 

(OPV  Radar  Heightsjl  3-1 9&1 6}+0.231 1 6)*OPV  Missile  Type[1] 

0.1190595 

0.024034 

4.95 

<.0001* 

(OPV  Radar  Heights{1 3-1 9&1 6J+0.231 1 6)*SFAC  Typefl] 

-0.108155 

0.023542 

-4.59 

<.0001* 

(OPV  Radar  Heightsjl  3-1 9&1 6J+0.231 1 6)*(OPVMaxSpeed-30.971 4) 

-0.018208 

0.004507 

-4.04 

<.0001* 

(OPV  Radar  Heights{22-1 3&1 9&1 6}+0.48744)*(OPVMaxSpeed-30.971 4) 

0.0175123 

0.004369 

4.01 

<.0001* 

(OPVMaxSpeed-30.971 4)*(OPVMaxSpeed-30.971 4) 

-0.003213 

0.000805 

-3.99 

<.0001* 

OPV  Radar  Heightsjl  9-1 6( 

-0.097176 

0.028021 

-3.47 

0.0007* 

(OPV  Radar  Heights{22-1 3&1 9&1 6}+0.48744)*SFAC  Typejl] 

0.07601 41 

0.022321 

3.41 

0.0008* 

Helo  Present[0] 

-0.060362 

0.019353 

-3.12 

0.0021* 

OPV  Radar  Heightsjl  3-1 9&1 6) 

-0.047509 

0.023605 

-2.01 

0.0457* 

Helo  Present[0]*OPV  Gun  Present[0] 

0.015705 

0.019765 

0.79 

0.4279 

Figure  50.  Combination  vs.  Enhance:  Avoidance  SFAC  Prosecution  Sorted 

Parameter  Estimates 


For  the  first  portion  of  the  combination  vs.  enhanced  OPV  scenario, 
avoidance  SFAC  prosecution  relies  more  on  the  type  of  SFACs  being  engaged,  and  the 
missile  system  equipped  on  the  OPVs.  As  in  the  previous  combination  scenario,  the 
majority  of  avoidance  SFACs  are  prosecuted  by  the  longer  range  missile  systems  rather 
than  the  naval  gun  or  griffin  missile  system.  The  reason  behind  this  is,  again,  that 
aggressive  SFACs  engage  the  OPVs.  Their  influence  in  the  simulation  environment 
serves  as  both  a  distraction  and  a  potential  threat  that  must  be  dealt  with  prior  to  OPVs 
continued  engagement  of  avoiding  SFACs.  Other  important  factors  to  note  are  OPV  max 
speed  and  radar  height  interactions  with  missile  and  SFAC  type.  These  three  factors  help 
to  position  OPVs  where  they  can  best  prosecute  the  SFAC  threat,  both  the  attacking  and 
evading  small  boats. 
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b.  Aggressive  SF AC  Prosecution  (Enhanced  OPVs) 


A  Sorted  Parameter  Estimates 

Tei  m 

Estimate 

Std  Error 

t  Ratio 

Pi  Ol)>|t| 

OPV  Radar  Heights{1 3-1 6&1 9&22} 

-0.003901 

0.000696 

-5.61 

<.0001* 

(OPV  Radar  Heights{1 3-1 6&1 9&22}+0.48744)*SFAC  Typed] 

-0.003636 

0.000696 

-5.23 

1  , 

:  ;  ; 

<.0001* 

SFAC  Type[1] 

-0.002269 

0.000607 

-3.74 

1 

0.0002* 

Figure  51.  Combination  vs.  Enhanced:  Aggressive  SFAC  Prosecution  Sorted 

Parameter  Estimates 


Mirroring  the  first  portion  of  the  results  shown  in  Figure  51,  the  parameter 
estimates  displayed  in  Figure  53  indicate  that  the  most  important  and  only  significant 
factor  in  determining  OPV  performance  for  aggressive  SFAC  prosecution  in  the 
combined  scenario  is  OPV  radar  height.  The  type  of  SFAC  engaged  is  also  significant, 
but  as  far  as  controllable  factors  for  the  OPV,  the  detection  capability  of  the  OPV, 
influenced  by  its  radar  height,  is  the  key  factor  to  consider.  This  result  reflects  many  of 
the  same  conditions  seen  in  the  Aggressive  SFAC  vs.  Enhanced  OPV  scenario  because 
OPV  max  speed,  SSM  equipped,  and  naval  gun  are  eclipsed  by  the  behavior  of  attacking 
SFACs  and  the  Griffin  missile  system.  The  helicopter  factor  is  negligible  as  well  since  all 
engagements  with  aggressive  SFACs  take  place  within  OPV  radar  detection  ranges,  so  it 
is  unnecessary  for  this  scenario. 


c.  Avoidance  and  Aggressive  SFAC  Prosecution  (Enhanced  OPVs) 


zi  Sorted  Parameter  Estimates 

Tenn 

Estimate 

Std  Ei  i  or 

t  Ratio 

Prob>|t| 

SFAC  Type[1] 

0.7702053 

0.019111 

40.30 

<.0001* 

OPV  Missile  Type[1] 

-0.440329 

0.019228 

-22.90 

:  I 

<.0001* 

OPVMaxSpeed 

0.0763699 

0.003748 

20.38 

I 

<.0001* 

OPV  Missile  Type[1]*SFAC  Type[1] 

0.2385031 

0.019641 

12.14 

zr 

<.0001* 

(OPV  Radar  Heights{22-1 3&1 9&1 6}+0.48744)*OPV  Missile  Type[1  ] 

-0.258481 

0.02271 1 

-11.38 

t 

<.0001* 

(OPV  Radar  Heights{1 9-1 6}+0.00503)*SFAC  Type[1  ] 

0.2192686 

0.02838 

7.73 

3 

<.0001* 

SFAC  Typefl  ]*(OPVMaxSpeed-30. 971 4) 

-0.027965 

0.003799 

-7.36 

<.0001* 

OPV  Radar  Heights{22-1 3&1 9&1 6} 

-0.154772 

0.02201 5 

-7.03 

<.0001* 

OPV  Gun  Present[0] 

-0.131564 

0.019435 

-6.77 

<.0001* 

(OPV  Radar  Heights{1 9-1 6}+0.00503)*OPV  Missile  Typefl] 

-0.144166 

0.028431 

-5.07 

L 

<.0001* 

(OPV  Radar  Heights{1 3-1 9&1 6}+0.231 1 6)*OPV  Missile  Typefl] 

0.1201348 

0.023995 

5.01 

] 

<.0001* 

(OPV  Radar  Heights{1 3-1 9&1 6}+0.231 1 6)*SFAC  Typefl  ] 

-0.111738 

0.023503 

-4.75 

<.0001* 

(OPV  Radar  Heights{1 3-1 9&1 6J+0.231 1 6)*(OPVMaxSpeed-30.971 4) 

-0.018129 

0.0045 

-4.03 

<.0001* 

(OPVMaxSpeed-30.971 4)*(OPVMaxSpeed-30.971 4) 

-0.003239 

0.000804 

-4.03 

<.0001* 

(OPV  Radar  Heights{22-1 3&1 9&1 6}+0.48744)*(OPVMaxSpeed-30.971 4) 

0.0175104 

0.004362 

4.01 

<.0001* 

(OPV  Radar  Heights{22-1 3&1 9&1 6}+0.48744)*SFAC  Typefl  ] 

0.0776727 

0.022284 

3.49 

0.0006* 

OPV  Radar  Heights{1 9-1 6} 

-0.096654 

0.027975 

-3.46 

0.0007* 

Helo  PresentfO] 

-0.060073 

0.019321 

-3.11 

0.0022* 

OPV  Radar  Heights{1 3-1 9&1 6} 

-0.051472 

0.023566 

-2.18 

0.0303* 

Helo  Present[0]*OPV  Gun  PresentfO] 

0.0160047 

0.019733 

0.81 

0.4184 

Figure  52.  Combination  vs.  Enhanced  Sorted  Parameter  Estimates 
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The  final  analysis  concerning  enhanced  OPV  factor  importance  is 
displayed  in  Figure  54,  and  it  is  almost  an  exact  replica  of  Figure  52.  What  this 
implies  is  that  the  avoidance  portion  of  the  combination  scenario  drives  the  factor 
importance  for  the  OPVs.  This  result  makes  sense  given  the  weak  factor 
significance  shown  in  Figure  53.  SFACs  employing  the  avoidance  behavior 
present  the  largest  challenge  for  OPVs  in  this  scenario,  so  it  is  the  factors  inherent 
to  that  portion  of  the  simulation  that  drive  the  results.  SFACs  employing  the 
aggressive  behavior  charge  the  OPVs  to  their  own  demise,  unable  to  impact  OPV 
behavior  for  long  due  to  the  effectiveness  of  the  ROC  behavior  provided  to  OPVs 
and  the  effectiveness  of  the  Griffin  missile  system. 

E.  FACTOR  DIFFERENCES 

1.  Naval  Gun 

In  contrast  to  the  baseline  results,  and  many  of  the  advanced  scenario  results 
utilizing  the  baseline  OPV  parameters,  the  naval  gun  loses  its  significance  in  the 
enhanced  OPV  models.  This  is  due  to  both  the  inclusion  of  weapons,  such  as  the  Griffin 
missile  which  is  specifically  designed  to  eliminate  the  SFAC  threat,  and  the  modeling  of 
the  naval  gun  to  more  accurately  reflect  the  effectiveness  of  naval  gunfire  at  sea.  During 
the  Second  World  War,  naval  gunfire  was  being  eclipsed  by  other  modes  of  combat  at 
sea.  Then  it  was  the  carrier  borne  attack  fighter,  capable  of  guiding  munitions  and 
torpedoes  at  far  greater  ranges  than  those  capable  by  guns  of  the  time.  Today  they  are 
further  diminished  in  their  use  by  radar  and  sensor  guided  munitions  which  can  have 
ranges  an  order  of  magnitude  higher  than  naval  gunfire,  and  are  far  more  accurate.  Naval 
gunfire  is  much  more  limited  than  missile  capabilities,  and  while  its  relevance  on  ships 
today  is  still  debated,  for  these  simulations  the  naval  gun  takes  second  seat  to  missile 
systems  such  as  the  Anti-Ship  Cruise  Missile  (ASCM)  or  self-defense  guided  missile  like 
the  Griffin. 

2.  Surface  to  Surface  Missile 

Unlike  the  decline  in  importance  shown  by  the  naval  gun,  equipped  missile  type 

continues  to  play  a  big  role  in  detennining  OPV  measures  of  performance.  The  long 
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range  and  high  probability  of  kill  outlined  in  the  provided  scenario  briefings  by  the  OSN 
staff  establish  a  weapon  system  very  capable  of  combating  a  small  boat  threat.  Its  only 
drawback  is  the  availability  of  ammunition  on  the  OPVs.  Missiles  like  the  Exocet  are 
very  large,  expensive,  and  their  employment  at  sea  takes  up  significant  deck  space  on 
current  naval  warships.  Since  these  models  did  not  address  the  ship  synthesis  problem 
inherent  with  fielding  ASCMs,  their  use  is  somewhat  questionable.  However,  their 
effectiveness  in  model  performance  is  not,  and  the  SSM  factor  remains  a  key  operational 
design  parameter  for  ship  measures  of  perfonnance. 

3.  OPV  Radar  Height 

Unlike  in  the  baseline  scenarios,  the  OPV  radar  height  factor  becomes  very 
significant  in  combating  the  avoidance  and  aggressive  SFAC  threat.  OPVs  facing  the 
swann  attack  of  suicide  SFACs  must  rely  on  organic  capabilities  to  defend  themselves  at 
times,  and  the  further  that  they  can  detect  this  threat  the  greater  their  survivability.  This 
importance  is  not  seen  so  much  in  the  individual  radar  height  factor,  but  in  the 
interactions  between  OPV  missile  type,  SFAC  type,  and  OPV  speed.  These  interactions 
help  to  paint  a  picture  that  the  OPV  radar  height  is  a  system  that  would  integrate  with 
many  different  facets  of  shipboard  operations  to  create  an  effective  fighting  vessel. 

4.  OPV  Max  Speed 

The  OPV  maximum  speed  factor  largely  remained  a  modest  contributor  to  OPV 
measures  of  performance.  Only  in  the  Avoidance  SFAC  vs.  Enhanced  OPV  and 
Combination  scenarios  did  OPV  maximum  speed  become  a  much  more  significant  factor. 
The  reason  behind  this  was  due  to  the  effectiveness  of  ROC  behavior  by  the  OPVs,  and 
the  Griffin  missile  system.  OPVs  could  easily  avoid  attacking  SFACs  in  the  combination 
scenario  and  subsequently  continue  to  pursue  evading  SFACs  to  increase  their  measure  of 
performance.  While  a  large  repertoire  of  weapon  systems  may  negate  the  need  of  a  quick 
and  agile  ship,  the  costs  associated  with  such  an  increase  may  be  prohibitive.  Abel’s 
thesis  concerning  frigate  survivability  illustrated  that  if  ships  were  quick  enough,  they 
could  evade  attacking  swarms  of  small  boats  (Abel,  2009),  thus  showing  how  important 
speed  can  be  to  ship  survival  in  an  ASUW  environment. 
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5.  Helicopter  Presence 

Much  like  in  the  OPV  radar  height  factor,  the  presence  of  a  helicopter  played  a 
moderate  role  in  the  OPV  measure  of  performance  of  SFAC  kills,  but  steadily  grew  in 
significance  for  the  enhanced  model  runs.  The  reason  behind  this  increase  in 
effectiveness  was  the  allowance  of  long  range  engagements  by  OP  Vs  at  the  onset  of  a 
simulation  run.  With  this  increase  in  detection  capability,  OPVs  could  quickly  eliminate  a 
large  number  of  the  SFAC  threat  prior  to  engaging  with  the  naval  gun  or  Griffin  missile 
system.  As  with  the  radar  height  factor,  interaction  tenns  with  helicopter  presence  such  as 
SFAC  type  and  SSM  type  became  prevalent  with  OPV  performance  since  OPVs  could 
maneuver  quickly  to  engage  the  SFAC  threat  early  in  the  simulation. 

6.  SFAC  Type 

The  type  of  threat  that  OPVs  were  facing  has  always  been  a  significant 
contributor  to  the  OPV  MOE  and  MOP.  In  both  the  baseline  and  advanced  model 
simulations,  the  parameters  changed  with  the  shifting  of  SFAC  type  from  one  to  the  next 
always  had  a  strong  influence.  Since  these  parameters  impacted  detection  range  and 
speed  at  which  SFACs  traveled,  they  had  conflicting  effects  at  times.  In  the  aggressive 
scenarios,  the  larger  SFAC  would  be  detected  at  longer  ranges,  thus  allowing  OPVs  to 
engage  them  earlier.  However,  their  increased  speed  had  a  counter  effect  on  OPV 
performance  because  these  same  SFACs  would  close  the  distance  quicker  on  OPVs  and 
possibly  eliminate  them  from  the  simulation.  For  the  avoidance  scenarios,  smaller  SFACs 
may  be  harder  to  detect  than  the  larger  variants,  but  their  reduced  speed  made  it  difficult 
for  them  to  avoid  pursuing  OPVs.  This  trade-off  between  detection  range  and  speed 
seemed  to  lean  toward  SFAC  speed  being  the  more  significant  detractor  from  OPV 
performance,  and  this  is  shown  in  various  interaction  terms  as  well  as  the  leverage  plots 
in  the  baseline  analysis. 
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V.  CONCLUSION 


A.  SUMMARY 

This  research  study  explored  the  simulation  modeling  effort  behind  operational 

effectiveness  models  which  provide  information  to  decision  makers  about  naval  ship 

design.  Focused  on  the  Anti-Surface  Warfare  scenario,  the  models  generated  during  the 
course  of  this  thesis  were  intended  to  determine  key  performance  factors  of  hypothetical 
Offshore  Patrol  Vessels  in  their  defense  against  the  small  boat  swarm  attack.  The 
simulation  platform  used  to  explore  this  work  was  MANA,  which  utilizes  an  influence 
system  to  alter  agent  behaviors  in  the  model.  These  behaviors  and  established  parameters 
were  varied  and  adjusted  to  help  create  an  understanding  of  which  design  factors  are  of 
key  importance  to  the  ever  evolving  and  changing  problem  of  the  small  boat  attack. 

B.  RESEARCH  QUESTIONS 

The  following  research  questions  were  presented  in  the  first  chapter  of  this  thesis: 

1 .  Based  on  the  limited  number  of  design  parameters  of  the  OPV  in  the  OSN 
model,  can  a  model  be  produced  in  the  MANA  environment  to  replicate 
similar  results? 

2.  Given  the  creation  of  this  initial  MANA  model  of  the  OPV’s  ASUW 
scenario,  can  additional  variables  be  introduced  to  determine  a  better 
Operational  Evaluation  Model  for  OPV  performance  and  subsequent  input 
into  the  Ship  Synthesis  Model  and  cost  models? 

With  the  analysis  conducted  in  Chapter  IV,  it  was  shown  that  it  was  indeed 
possible  to  replicate  a  model  in  MANA  which  behaved  very  similarly  to  the  OSN  agent 
based  model.  While  slight  differences  were  present,  over  98%  of  all  variation  between 
the  two  models  was  accounted  for,  and  all  interactions  terms  with  the  Indicator[Italian] 
variable  were  deemed  insignificant,  and  therefore  removed. 
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The  results  generated  by  having  an  interchangeable  MANA  baseline  model 
helped  to  create  a  basis  for  comparison  between  baseline  factor  significance  to  later 
detennine  advanced  model  results. 

Addressing  the  second  question,  it  was  shown  through  a  series  of  model 
improvements  that  key  detenninants  of  model  perfonnance  did  change,  and  that  the 
resulting  insights  created  a  more  realistic  set  of  factors  for  decision  makers  to  focus  on  in 
ship  development.  These  factors  agree  with  findings  from  historical  records  concerning 
the  small  boat  swann  threat,  and  support  current  published  naval  doctrine  by  nations  in 
highly  tense  regions  of  the  world. 

C.  RECOMMENDATIONS  &  FUTURE  WORK 

Given  the  constrained  modeling  parameters  provided  by  the  OSN  ASUW 
scenario,  exploration  of  a  larger  set  of  decision  factors  that  influence  measures  of 
effectiveness  and  measures  of  performance  may  be  of  significant  worth.  This  study  was 
part  of  an  effort  by  students  and  faculty  at  NPS  to  support  and  verify  practices  undertaken 
by  OSN,  as  directed  by  ONR.  Expanding  upon  the  design  space  of  the  models  created  in 
this  thesis  may  help  to  gather  better  insight  on  key  perfonnance  factors  which  can  then  in 
turn  be  placed  into  decision  tools  to  aid  in  ship  design.  Additionally,  incorporating  more 
real  world  systems  and  characteristics  of  modern  naval  warships  may  also  be  of  use.  To 
further  the  realism  of  the  scenarios  presented,  the  inclusion  of  noise  factors,  such  as  sea- 
state,  weather,  and  obstacles,  may  also  improve  understanding  of  this  complex  problem. 
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